Arab World English Journal (AWEJ) Volume 12. Number2 June 2021 Pp. 49 – 71
Validating English Language Entrance Test at a Saudi University for Health Sciences
Department of English, College of Science and Health Professions, King Saud bin Abdulaziz University for Health Sciences & King Abdullah International Medical Research Center, Jeddah, Saudi Arabia
Manal Al Makoshi
Department of English, College of Science and Health Professions, King Saud bin Abdulaziz University for Health Sciences & King Abdullah International Medical Research Center, Riyadh, Saudi Arabia. Correspondent Author: firstname.lastname@example.org
Department of English, College of Science and Health Professions, King Saud bin Abdulaziz University for Health Sciences & King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
Department of Pathology, College of Medicine, King Saud bin Abdulaziz University for Health Sciences & King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
Received: 2/23/2021 Accepted: 4/10/2021 Published:6/24/2021
This paper aims to validate the English Language Entrance Test for King Saud bin Abdulaziz University for Health Science (hereafter KSAU-ET). It supports the argument regarding using specially designed in-house entrance tests at health universities by showcasing the test’s development, administration, and validation process. It presents a new framework for test validation that is informed by various existing frameworks such as Messick (1996), Sireci (1998) and, Weir (2005), with a specific focus on the notions of unitary and practicality. The proposed framework treats validity as a pre-, during, and post-test process that collects evidence from each phase to support the test’s overall validity. The data were collected using different tools and through the three stages of the validation process. The test was taken by 474 candidates who applied to join KSAU-HS Stream II medical program. The data confirmed that the test was reliable (alpha > 7) and reasonably meet the university’s needs to select the program’s top prospective candidates. Nevertheless, the study highlighted the importance of collecting further evidence in future studies and including more selection criteria in the regression model of analysis. Using this framework, the study contributes to the existing body of research that investigate English entrance test validation. It shows that exam validity is a context-sensitive process strongly associated with the purpose for which the exam is used. Finally, the paper discusses pedagogical implications that may help educators at health science universities develop in-house entrance tests in place of standardized tests, which often do not address context, curriculum, or program objectives.
Keywords: English entrance test, health science, language assessment validity, practicality, validation, King Saud bin Abdulaziz University for Health Sciences
Cite as: Jawhar, S., Al Makoshi,M., Alhawsawi, S., & Alkushi, A. (2021). Validating English Language Entrance Test at a Saudi University for Health Sciences. Arab World English Journal, 12 (2) 49-71.
Ali, M. A. (1987). Effect of sample size on the size of the coefficient of determination in simple linear regression. Journal of Information and Optimization Sciences, 8(2), 209-219.
Allen, M.J., & Yen, W.M. (2002). Introduction to measurement theory. Long Grove, IL: Waveland Press.
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford, UK: Oxford University Press.
Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice: Developing language assessments and justifying their use in the real world. Oxford, UK: Oxford University Press.
Brookhart, S., & Nitko, A. (2019). Educational Assessment of Students plus with MyLab Education with Pearson eText. London, UK: Pearson.
Brown, J. D. (1993). A comprehensive criterion-referenced language testing project. In D. Douglas, & C. Chapelle (Eds.), A new decade of language testing research: Selected Papers from the Annual Language Testing Research Colloquium (pp. 163-184). Washington, DC, USA: TESOL.
Brown, J. D. (1996). Testing in language programs. Upper Saddle River, NJ: Prentice Hall.
Chapelle, C. A., Enright, M., & Jamieson, J. (2008). Building a validity argument for the Test of English as a Foreign Language. London: Routledge.
Crocker, L.M., Miller, D., & Franks, E.A. (1989). Quantitative methods for assessing the fit between test and curriculum. Applied Measurement in Education, 2(2), 179-194. https://doi.org/10.1207/s15324818ame0202_6
Cronbach, L.J., & Meehl, P.E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302. https://doi.org/10.1037/h0040957
Deanship of Admissions and Registration (2019). Admission Requirements. Retrieved from https://www.ksau-hs.edu.sa/English/Admission/Pages/AdmissionRequirements.aspx
Dinh, M. T. (2019). A review on validating language tests. VNU Journal of Foreign Studies, 35(1),143-154. DOI: https://doi.org/10.25073/2525-2445/vnufs.4343
Ebel, R. L. (1966). Measuring educational achievement. India: Prentice Hall.
Fulcher, G. (1997). An English language placement test: issues in reliability and validity. Language Testing, 14(2), 113–139.
Gardner, S., Liu, P., & Roberts‐Thomson, K. (2020). Trajectory of performance; the role of selection criteria on student achievement in a Bachelor of Oral Health program. European Journal of Dental Education, 24(3), 572-579.
Ginther, A., & Yan, X. (2018). Interpreting the relationships between TOEFL iBT scores and GPA: Language proficiency, policy, and profiles. Language Testing, 35(2), 271-295.
Guan, T., Alam, M. K., & Rao, M. B. (2019). Sample Size Calculations in Simple Linear Regression: Trials and Tribulations. arXiv preprint arXiv:1907.10569, 1-20
Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied measurement in education, 2(1), 37-50.
Hamouda, A. (2013). An Investigation of Listening Comprehension Problems Encountered by Saudi Students in the EL Listening Classroom. International Journal of Academic Research in Progressive Education and Development, 2(2), 113-155.
Im, GH., Shin, D., & Cheng, L (2019) Critical review of validation models and practices in language testing: their limitations and future directions for validation research. Lang Test Asia, 9(14). https://doi.org/10.1186/s40468-019-0089-4
Inoue, N. (2006). What’s going on inside the pine tower of babel: Foreign language curriculum reform in a Japanese university. Languages and Cultures Series, 16, 87-115.
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73. DOI: https://doi.org/10.1111/jedm.12000
Kane, M., Crooks, T., & Cohen, A. (1999). Validating measures of performance. Educational measurement: issues and practice, 18(2), 5-17.
Messick, S. (1996). Validity and washback in language testing. Language testing, 13(3), 241-256. DOI: https://doi.org/10.1177/026553229601300302
Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational researcher, 18(2), 5-11.
Miller, J. D., et al. (2013). The Five-Factor Narcissism Inventory (FFNI): A test of the convergent, discriminant, and incremental validity of FFNI scores in clinical and community samples. Psychological Assessment, 25(3), 748-758. DOI: 10.1037/a0032536
Ozer, I., Fitzgerald, S. M., Sulbaran, E., & Garvey, D. (2014). Reliability and content validity of an English as a foreign language (EFL) grade-level test for Turkish primary grade students. Procedia-Social and Behavioral Sciences, 112, 924-929.
Park, E. (2019). The Effects of Linguistic and Demographic Features of Chinese International Students on Placement Test Levels in Higher Education: Logistic Regression. Journal of International Students, 9(1), 225–241.
Puddey, I. B., & Mercer, A. (2014). Predicting academic outcomes in an Australian graduate entry medical programme. BMC Medical Education, 14(1), 31. DOI: https://doi.org/10.1186/1472-6920-14-31
Ramsey, P. A. (1993). Sensitivity review: The ETS experience as a case study. Differential item functioning, 38(2), 367-388.
Sireci, S.G. (1998). The construct of content validity. Social Indicators Research, 45, 83-117. DOI: 10.1023/A:1006985528729
Sireci, S., & Faulkner-Bond, M. (2014). Validity evidence based on test content. Psicothema, 26(1), 100-107. DOI: 10.7334/psicothema2013.256
Wall, D., Clapham, C., & Alderson, J. C. (1994). Evaluating a placement test. Language Testing, 11(3), 321-344.
Weir, C. J. (2005). Language testing and validation. Hampshire: Palgrave McMillan.
Westrick, P. (2005). Score Reliability and Placement Testing. JALT Journal, 27(1), 71-92.
Xi, X., & Sawaki, Y (2017) Methods of Test Validation. In E. Shohamy, I. G. Or, & S. May, (Eds.), Language testing and assessment, (pp. 193–210). United States: Springer International Publishing.
Zwick, R. (2002). Is the SAT a ‘wealth test’? Phi Delta Kappan, 84(4), 307-311.