Arab World English Journal (AWEJ) Volume 11. Number4 December 2020 Pp.363-381
Standardization of EFL Undergraduate Skill Exam Papers: A Case Study at Majmaah
Majed Othman Abahussain
College of Education, Majmaah University, Saudi Arabia
College of Education, Majmaah University, Saudi Arabia
College of Education, Majmaah University, Saudi Arabia
Designing standardized exams for EFL undergraduates is a matter of great concern in pedagogical practices that assess learners’ academic achievements. This case study was conducted at Majmaah University, Saudi Arabia, to explore the significance of educational exam standardization and its impact on the process of implementation, leading to a balanced and equitable assessment of English language skills. During the study, 250 final English skill exam questions of two semesters were analyzed. A mixed-method approach using a questionnaire and Focused Group Discussion was adopted to collect relevant data. The quantitative data were analyzed using SPSS 16, and the qualitative data was processed by thematic analysis. The quantitative data results showed that questions were balanced, valid, and reliable in language skill exam papers designed for undergraduate English students. The qualitative data revealed four major themes – standardization of questions, the efficacy of Bloom’s taxonomy, assessment/evaluation, and alignment of the question with learning outcomes. These emerging themes highlighted the effectivness of standardizing exam papers, identified problems, and suggested ways of improvement. The study indicates that exams must be scrutinized for the sake of standardization in terms of maximum course coverage; the variety in the question item; clear, concise, and precise instructions; validity and reliability of exam papers; and the alignment of questions with the course and program learning outcomes to ensure the implementation of the revised Bloom’s taxonomy. The study suggests, to generalize the results, that further investigation should be done in another Saudi university for the sake of comparison.
Keywords: Standardization, EFL undergraduate students, Majmaah University, Bloom’s taxonomy, Course Learning Outcomes
Cite as: Abahussain,M. O., Iqbal,M., & Khan, K. (2020). Standardization of EFL Undergraduate Skill Exam Papers: A Case Study at Majmaah University Arab World English Journal, 11 (4) 363-381.
Adams, N. E. (2015). Bloom’s taxonomy of cognitive learning objectives. Journal of the Medical Library Association: JMLA, 103(3), 152-153. doi: 10.3163/1536-5050.103.3.010
Airasian, P. W., & Miranda, H. (2002). The role of assessment in the revised taxonomy. Theory into practice, 41(4), 249-254. https://doi.org/10.1207/s15430421tip4104_8
Alfallaj, F. S. S., & Al-Ahdal, A. A. M. H. (2017). Authentic assessment: Evaluating the Saudi EFL tertiary examination system. Theory and Practice in Language Studies, 7(8), 597-607. https://doi.org/10.17507/tpls.0708.01
Alzu’bi, M. A. (2014). The Extent of Adaptation Bloom’s Taxonomy of Cognitive Domain in English Questions Included in General Secondary Exams. Advances in Language and Literary Studies, 5(2), 67-72. https://doi.org/10.7575/aiac.alls.v.5n.2p.67
Anderson, L. W. (2002). Curricular alignment: A re-examination. Theory into practice, 41(4), 255-260. https://doi.org/10.1207/s15430421tip4104_9
Ashadi, R. I., & Lubis, N. (2017). A Survey on the Levels of Questioning of ELT: A Case Study in an Indonesian Tertiary Education. Advances in Language and Literary Studies, 8(3), 26-31.
Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30(1), 3-12. https://doi.org/10.1111/j.1745-3992.2010.00195.x
Brown, S. & Race, P. (2012). Using Effective Assessment to Promote Learning. In L. Hunt, & D. Chalmers, (Eds.), University Teaching in Focus: a learning-centred approach (pp. 74-91). Camberwell, VIC: ACER.
Brown, J. D. (2019). 5 Global Englishes and the International Standardized English Language Proficiency Tests. Critical Perspectives on Global Englishes in Asia: Language Policy, Curriculum, Pedagogy and Assessment, 64.
Black, P., & William, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 21(1), 5-31.
Bates, C., & Hopkins, A. (2007). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education), 21, 5-31. https://doi.org/10.1007/s11092-008-9068-5
Crespo, R. M. et al. (2010). Aligning assessment with learning outcomes in outcome-based education. In IEEE EDUCON 2010 Conference (pp. 1239-1246). IEEE. https://doi.org/10.1109/EDUCON.2010.5492385
Downing, S. M. (2005). The effects of violating standard item writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in health sciences education, 10(2), 133-143. https://doi.org/10.1007/s10459-004-4019-5
Dunbar, N. E., Brooks, C. F., & Kubicka-Miller, T. (2006). Oral communication skills in higher education: Using a performance-based evaluation rubric to assess communication skills. Innovative Higher Education, 31(2), 115-128. doi: 10.1007/s10755-006-9012-x
Estes, A. C., Welch, R. W., & Ressler, S. J. (2004). Questioning: Bring your students along on the journey. Journal of Professional Issues in Engineering Education and Practice, 130(4), 237-242. https://doi.org/10.1061/(ASCE)1052-3928(2004)130:4(237)
Fathelrahman, A. (2019). Measuring the Effectiveness of Holistic Approach to Teaching and Assessment used in Teaching a Decision Support Systems (DSS) Course. https://doi.org/10.31124/advance.9771806.v1
Forehand, M. (2010). Bloom’s taxonomy. Emerging perspectives on learning, teaching, and technology, 41(4), 47-56. Retrieved from: https://www.d41.org/cms/lib/IL01904672/Centricity/Domain/422/BloomsTaxonomy.pdf
Giraldo, F. (2018). Language assessment literacy: Implications for language teachers. Profile Issues in TeachersProfessional Development, 20(1), 179-195.Hendrickson, K. A. (2012). Assessment in Finland: A Scholarly Reflection on One Country’s Use of Formative, Summative, and Evaluative Practices. Mid-Western Educational Researcher, 25(1-2), 33-43. Retrieved from.https://www.mwera.org/MWER/volumes/v25/issue1-2/v25n1-2-.pdf
Hamad, M. M., & Alnuzaili, E. S. (2020). Steps of Designing a Personal Questions Bank in a Pedagogical Way. Theory and Practice in Language Studies, 10(6), 638-646.
He, L., & Shi, L. (2008). ESL students’ perceptions and experiences of standardized English writing tests. Assessing Writing, 13(2), 130-149. https://doi-org.sdl.idm.oclc.org/10.1016/j.asw.2008.08.001
Holme, T. (2003). Assessment and quality control in chemistry education. Journal of Chemical Education, 80(6), 594-596. https://doi.org/10.1021/ed080p594
Inbar‐Lourie, O. (2012). Language assessment literacy. The encyclopedia of applied linguistics. https://doi.org/10.1002/9781405198431.wbeal0605
Jenkins, J., & Leung, C. (2017). Assessing English as a lingua franca. Language testing and assessment, 103-117.
Jovanovska, J. (2018). Designing effective multiple-choice questions for assessing learning outcomes. Infotheca, 18(1), 25-42.
Karamustafaoğlu, S., Sevim, S., Karamustafaoğlu, O., & Çepni, S. (2003). Analysis of Turkish high-school chemistry-examination questions according to Bloom’s taxonomy. Chemistry Education Research and Practice, 4(1), 25-30. https://doi.org/10.1039/B2RP90034C
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into practice, 41(4), 212-218. https://doi.org/10.1207/s15430421tip4104_2
Krathwohl, D. R., & Anderson, L. W. (2009). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.
Krippendorff, K. (2018). Content analysis: An introduction to its methodology. Sage public
Köksal, D., & Ulum, Ö. G. (2018). Language assessment through Bloom’s Taxonomy. Journal of language and linguistic studies, 14(2), 76-88.
Kunen, S., Cohen, R., & Solman, R. (1981). A levels-of-processing analysis of Bloom’s taxonomy. Journal of Educational Psychology, 73(2), 202-211. https://doi.org/10.1037/0022-06220.127.116.11
Lach, M. C. (2014). Teacher-Generated Final Exams in High School Science: Content, Rigor, and Assessment Literacy, (Unpublished Doctoral dissertation). Retrieved from http://www.indigo.lib.uic.edu:8080/bitstream/handle/10027/18824/Lach_Michael.pdf
Malik, S. I. (2019). Assessing the teaching and learning process of an introductory programming course with Bloom’s taxonomy and Assurance of Learning (AOL). +International Journal of Information and Communication Technology Education (IJICTE), 15(2), 130-145.
National Center for Academic Accreditation and Assessment. (2018, October 10). Retrieved October 31, 2019, from https://ncaaa.org.sa/enportal/Pages/default.aspx.
NCAAA. (2019). Handbook for NCAAA Key Performance Indicators. National Commission for Academic Accreditation & Assessment, Riyadh, Kingdom of Saudi Arabia. Retrieved from https://ncaaa.org.sa/enportal/Pages/default.aspx
Nusche, D. (2008). Assessment of Learning. Paris: OECD publishing. doi:10.1787/244257272573
Ounis, A. (2017). The assessment of speaking skills at the tertiary level. International Journal of English Linguistics, 7(4), p95.
Qasrawi, R., & BeniAbdelrahman, A. (2020). The Higher and Lower-Order Thinking Skills (HOTS and LOTS) in Unlock English Textbooks (1st and 2nd Editions) Based on Bloom’s Taxonomy: An Analysis Study. International Online Journal of Education and Teaching, 7(3), 744-758.
Serpil, Ö. Z., & Derin, A. T. A. Y. (2017). Turkish EFL instructors’ in-class language assessment literacy: Perceptions and practices. ELT Research Journal, 6(1), 25-44.
Sivaraman, S. I., & Krishna, D. (2015). Blooms taxonomy–application in exam papers assessment. Chemical Engineering (VITU), 12(12), 32. Retrieved from https://www.researchgate.net/profile/Ilango_Sivaraman/publication/285883763_Blooms_Taxonomy-_Application_in_Exam_Papers_Assessment/links/56641aaf08ae418a786d32d4/Blooms-Taxonomy-Application-in-Exam-Papers-Assessment.pdf
Staub, D., & Kirkgöz, Y. (2019). Standards Assessment in English Language Teacher Education. Novitas-ROYAL (Research on Youth and Language), 13(1), 47-61. Retrieved from https://files.eric.ed.gov/fulltext/EJ1214199.pdf
Swart, A. J. (2009). Evaluation of final examination papers in engineering: A case study using Bloom’s Taxonomy. IEEE Transactions on Education, 53(2), 257-264. https://doi.org/10.1109/TE.2009.2014221
Tajgozari, M., & Alimorad, Z. (2019). Iranian EFL students’ perceptions of criteria for assessing students’ written performance. Global Journal of Foreign Language Teaching, 9(1), 2-9.
Taras, M. (2005). Assessment–summative and formative–some theoretical reflections. British journal of educational studies, 53(4), 466-478. https://doi.org/10.1111/j.1467-8527.2005.00307.x
Tsang, A. (2020). The relationship between tertiary-level students’ self-perceived presentation delivery and public speaking anxiety: A mixed-methods study. Assessment & Evaluation in Higher Education, 1-13.
Vella Briffa, P. (2013). Explicating an English speaking examination: Challenges and considerations. Symposia Melitensia, 9, 193-208. Retrieved from https://www.um.edu.mt/library/oar/handle/123456789/8748
Wilkinson, S. (2004). Focus groups. In Breakwell, G. M. (ed.), Doing social psychology research (pp. 344-376). The British Psychological Society and Blackwell Publishing Ltd https://doi.org/10.1002/9780470776278.ch14
Wozney, L., Venkatesh, V., & Abrami, P. (2006). Implementing computer technologies: Teachers’ perceptions and practices. Journal of Technology and teacher education, 14(1), 173-207. Retrieved from https://www.learntechlib.org/p/5437/