Arab World English Journal (AWEJ) Volume 14. Number 4 December 2023                   Pp.213-232

Full Paper PDF

Topic Familiarity Effects on Performance in Speaking Assessment Tasks

Nurul Iman Ahmad Bukhari
Faculty of Language Studies and Human Development, Universiti Malaysia Kelantan, Kelantan, Malaysia
Faculty of Educational Studies, Universiti Putra Malaysia, Selangor, Malaysia

Lilliati Ismail
Faculty of Educational Studies, Universiti Putra Malaysia, Selangor, Malaysia
Corresponding Author:

 Noor Lide Abu Kassim
Kuliyyah of Education, International Islamic University Malaysia, Kuala Lumpur, Malaysia
Corresponding Author:

Abu Bakar Razali
Faculty of Educational Studies, Universiti Putra Malaysia, Selangor, Malaysia

 Nooreen Noordin
Faculty of Educational Studies, Universiti Putra Malaysia, Selangor, Malaysia

 Muhamad Firdaus Mohd Noh
Sekolah Rendah Agama Bersepadu Segamat, Johor, Malaysia


Received:07/11/2023                                Accepted: 11/28/2023                             Published:  12/15/2023


Speaking assessment is believed to be difficult in its expansion and execution.   Thus, it is a challenge for teachers in preparing students for speaking tests. This study’s purpose was to identify whether topic familiarity affects speaking performance among students who were preparing to sit for the high-stakes Malaysian University English Test speaking test.  The study aimed to investigate the validity and reliability of the speaking assessment measures and, subsequently, to examine the speaking tests’ item difficulty measures differences according to topic familiarity level.  Data were collected from 40 non-native speakers of English among Malaysian Form Six pre-university students who were preparing for their MUET test. The researcher conducted two practice speaking tests, which used retired papers of the MUET CEFR-aligned speaking tests, to the 40 participants who were grouped into 10 speaking test groups. The practice speaking tests were video and audio-recorded. Topic familiarity was measured using self-report questionnaires.  In the second phase, each of the seven appointed raters scored all 40 students’ responses in two speaking tests consisting of two speaking tasks assessed across four criteria: task fulfillment, language, communicative ability, and group discussion.  Many-facet Rasch measurement results revealed significant differences translating to significant influence of topic familiarity on speaking performance. The present study’s results not only confirmed the significance of topic familiarity in the preparation for speaking assessments, but also highlighted the need for formal teaching on potential topic themes that are commonly encountered in such assessments, particularly those that are at the post-secondary level.  These findings imply importance in designing speaking tests taking into account test-taker topic familiarity.
Keywords: language testing, many-facet Rasch measurement, oral performance, speaking assessment, topic familiarity

Cite as:  Bukhari, N. I.A., Ismail, L., Abu Kassim, N.L., Abu Bakar Razali, A., Noordin, N., & Noh, M. F. M. (2023). Topic Familiarity Effects on Performance in Speaking Assessment Tasks.  Arab World English Journal, 14 (4) 213-232.


Abu Bakar, N. I., Noordin, N., & Razali, A. B. (2019). Improving Oral Communicative Competence in English Using Project-Based Learning Activities. English Language Teaching, 12(4), 73–84.

Abu Kassim, N. L., & Zubairi, A. M. (2006). Interaction Between Test-taker Characteristics, Task Facets and L2 Oral Proficiency Test Performance. Educational Awakening: Journal of the Educational Sciences, 3(2), 139–159.

Afaf Ayed Alrowaithy. (2021). The Effect of Topic Familiarity on Improving EFL Saudi Female Students` Reading Comprehension. Journal of Educational and Psychological Sciences, 5(7), 118–135.

Ahmad Tarmizi, S. A., et al. (2022). A Socio-Cognitive Perspective on the Factors Affecting Malaysian Business Students’ Learning when Spoken in English in a Second-Language Classroom. International Journal of Learning, Teaching and Educational Research, 21(1), 67–91.

Ameri-Golestan, A. (2016). Effects of Teaching Strategies and Topic Familiarity on Persian Speaking IELTS Candidates’ Speaking Scores. Modern Journal of Language Teaching Methods, 6(3), 44–50.

Anselmi, P., Colledani, D., & Robusto, E. (2019). A Comparison of Classical and Modern Measures of Internal Consistency.
Frontiers in Psychology, 10(December), 1–12.

Aryadoust, V., Ng, L. Y., & Sayama, H. (2021). A comprehensive review of Rasch measurement in language assessment: Recommendations and guidelines for research. Language Testing, 38(1), 6–40.

Bachman, L. F. (1999). Appendix: Language testing – SLA research interfaces. In L. F. Bachman & A. D. Cohen (Eds.), Interfaces between Second Language Acquisition and Language Testing Research (pp. 177–195). Cambridge University Press.

Badrasawi, K. J. I., Kassim, N. L. A., Zubairi, A. M., Johar, E. M., & Sidik, S. S. (2021). English Language Speaking Anxiety, Self-Confidence and Perceived Ability among Science and Technology Undergraduate Students: A Rasch Analysis. Pertanika Journal of Social Sciences and Humanities, 29(3), 309–334.

Baharudin, H., Maskor, Z. M., & Matore, M. E. E. M. (2022). The raters’ differences in Arabic writing rubrics through the Many-Facet Rasch measurement model. Frontiers in Psychology, 13, 7760.

Banerjee, H. L. (2019). Investigating the Construct of Topical Knowledge in Second Language Assessment: A Scenario-Based Assessment Approach. Language Assessment Quarterly, 16(2), 133–160.

Bijani, H. (2018). Effectiveness of a Face-to-Face Training Program on Oral Performance Assessment : The Analysis of Tasks Using the Multifaceted Rasch Analysis. 5(4), 27–53.

Bijani, H. (2019). Evaluating the effectiveness of the training program on direct and semi-direct oral proficiency assessment: A case of multifaceted Rasch analysis. Cogent Education, 6(1), 1670592.

Bijani, H., & Khabiri, M. (2017). Investigating the effect of training on raters’ bias toward test takers in oral proficiency assessment: A FACETS analysis. Journal of Asia TEFL, 14(4), 687–702.

Bond, T. G., & Fox, C. (2015). Applying the Rasch Model; Fundamental Measurement in the Human Sciences |
Request PDF (3rd ed.). Routledge.

Bonk, W. J., & Ockey, G. J. (2003). A many-facet Rasch analysis of the second language group oral discussion task.
Language Testing, 20(1), 89–110.

Brown, A. (2003). Interviewer variation and the co-construction of speaking proficiency. Language Testing, 20(1), 1–25.

Bui, G., & Huang, Z. (2016). L2 fluency as influenced by content familiarity and planning : Performance , measurement ,
and pedagogy. Language Teaching Research, 22(1), 94–114.

Chonghui, L. (2019). Boosting English Standards. The Star Online. Available at

Council of Europe. (2001). the Common European Framework of Reference for Languages : Learning, Teaching, Assessment.
Council of Europe.

Fischer, J. (2020). The underlying action-oriented and task-based approach of the CEFR and its implementation in language testing and assessment at university. Language Learning in Higher Education, 10(2), 301–316.

Fu, J. S., Yang, S. H., & Yeh, H. C. (2021). Exploring the impacts of digital storytelling on English as a foreign language learners’ speaking competence. Journal of Research on Technology in Education, 0(0), 1–16.

Geranpayeh, A., & Ahmad Zufrie, A. R. (2018). The Alignment Of Malaysian University English Test To The CEFR. 6th British Council New Directions in English Language Assessment Conference: Standards in Learning Systems, October, 21. Retreived from

Han, L., & Md Yusof, M. A. (2019). Students’ Self-reflections of Own Participation in English Language Oral Class. LSP International Journal, 6(2), 79–92.

Hidri, S. (2018). Assessing spoken language ability: A many-facet Rasch analysis. Second Language Learning and Teaching, 9783319628837, 23–48.

Huang, H. T. D., Hung, S. T. A., & Hong, H. T. V. (2016). Test-Taker Characteristics and Integrated Speaking Test Performance: A Path-Analytic Study. Language Assessment Quarterly, 13(4), 283–301.

Huang, H. T. D., Hung, S. T. A., & Plakans, L. (2018). Topical knowledge in L2 speaking assessment: Comparing independent and integrated speaking test tasks. Language Testing, 35(1), 27–49.

Idris, M. B., & Abdul Raof, A. H. Bin. (2019). Learner-Driven Oral Assessment Criteria for English Presentation
. Journal of Nusantara Studies (JONUS), 4(1), 365.

Idris, M., & Zakaria, M. H. (2016). Gauging esl learners’ cefr ratings on oral proficiency in rater training. Man in India,
96(6), 1675–1682. Available at,5&scioq=GAUGING+ESL+LEARNERS’+

Ismail, L., & Abd. Samad, A. (2014). Using Tasks and Repair Practices in the Malaysian L2 Classroom. The Asia-Pacific Education Researcher, 23(3), 499–510.

Jahedi, M., & Ismail, L. (2020). Factors affecting ESL students’ willingness to communicate in english classroom
discussions and their use of linguistic strategies. Universal Journal of Educational Research, 8(8), 3360–3370.

Khabbazbashi, N. (2017). Topic and background knowledge effects on performance in speaking assessment. Language
Testing, 34(1), 23–48.

Lai Kuen, G., Rafik-Galea, S., & Swee Heng, C. (2017). Effect of Oral Communication Strategies Training on the Development of Malaysian English as a Second Language Learners’ Strategic Competence. International Journal of Education and Literacy Studies, 5(4), 57.

Lateh, N. H. M., Shamsudin, S., & Said, S. M. (2015). Influence of Malaysian University English Test Training on the Speaking Performance of Pre-University Students. Advanced Science Letters, 21(7), 2463–2465.

Linacre, J. M. (2023). Facets computer program for many facet Rasch measurement (Version 3.87.0).

Lumley, T., & O’Sullivan, B. (2005). The effect of test-taker gender, audience and topic on task performance in tape-mediated assessment of speaking. Language Testing, 22(4), 415–437.

Malaysian Examinations Council. (2019). Malaysian University English Test (MUET). In Batu Caves: Percetakan Warni. Retreived from

Mohd Noh, M. F. Bin, & Mohd Matore, M. E. E. Bin. (2020). Rating performance among raters of different experience
through multi-facet rasch measurement (MFRM) model. Journal of Measurement and Evaluation in Education and
Psychology, 11(2), 147–162.

Mohd Noh, M. F., & Mohd Matore, M. E. E. (2022). Rater severity differences in English language as a second language speaking assessment based on rating experience, training experience, and teaching experience through many-faceted Rasch measurement analysis. Frontiers in Psychology, 13, 1-13

O’Grady, S. (2019). The impact of pre-task planning on speaking test performance for English-medium university
admission. Language Testing, 36(4), 505–526.

Ovilia, R. (2019). The Relationship of Topic Familiarity and Listening Comprehension. 276(Icoelt 2018), 182–186.

Qiu, X. (2019). Functions of oral monologic tasks: Effects of topic familiarity on L2 speaking performance. Language Teaching Research. 24(6), 745-764.

Qiu, X., & Lo, Y. Y. (2016). Content familiarity, task repetition and Chinese EFL learners’ engagement in second language use. Language Teaching Research, 21(6), 681–698.

Rethinasamy, S., & Chuah, K. M. (2011). The Malaysian university English test (MUET) and its use for placement purposes: A predictive validity study. Electronic Journal of Foreign Language Teaching, 8(2), 234–245.

Sims, M. E., Cox, T. L., Eckstein, G. T., Hartshorn, K. J., Wilcox, M. P., & Hart, J. M. (2020). Rubric Rating with MFRM versus Randomly Distributed Comparative Judgment: A Comparison of Two Approaches to Second-Language Writing Assessment. Educational Measurement: Issues and Practice, 39(4), 30–40.

Skehan, P. (2014). Processing Perspectives on Task Performance. John Benjamins Publishing Company.

Tahsildar, M. N., & Yusoff, Z. S. (2014). Investigating L2 students’ listening anxiety: A survey at a Malaysian university. International Journal of Language Education and Applied Linguistics, 1, 43–52.

Weir, C. J., & Wu, J. R. W. (2006). Establishing test form and individual task comparability: A case study of a semi-direct speaking test. Language Testing, 23(2), 167–197.

Xiaolei, S., Ismail, L., Yurong, H., & Mengqi, W. (2023). Strategies of Content Knowledge Representation and EFL Learners’ English Writing Proficiency: Mediating Role of Critical Thinking Skills. Arab World English Journal, 14(3), 309–323.

Xu, J., & Qiu, X. (2021). Engaging L2 Learners in Information-gap Tasks: How Task Type and Topic Familiarity Affect Learner Engagement. RELC Journal, 0(0).

Received: 07/11/2023  
Accepted: 11/28/2023 
Published: 12/15/2023   

Nurul Iman Ahmad Bukhari is teaching English at Universiti Malaysia Kelantan. She is also currently a PhD TESL candidate at Universiti Putra Malaysia.  Her research interests include ESL speaking performance and language assessment.

Lilliati Ismail, Ph. D. is a Senior Lecturer at the Faculty of Educational Studies, Universiti Putra Malaysia. Her research interests include grammar instruction and task-based language teaching.

Noor Lide Abu Kassim, Ph.D. is a Professor at the Kuliyyah of Education, International Islamic University Malaysia. Her research interests include Psychometrics and Education Evaluation.

Abu Bakar Razali, Ph.D. is an Associate Professor at the Faculty of Educational Studies, Universiti Putra Malaysia. His research interests include teaching of English as a second language (TESL), literacy and language education.

Nooreen Noordin, Ph.D. is an Associate Professor at the Faculty of Educational Studies, Universiti Putra Malaysia. Her research interests include content-based instruction, technology-enhanced language learning, learning styles, motivation, and best teaching practices in ESL.

Muhamad Firdaus Mohd Noh is an English primary school teacher. He is currently a PhD candidate at Universiti Kebangsaan Malaysia. His research interests include psychometrics and language assessment.