Integrating Subtitles for the Deaf and Hard of Hearing (SDH) into Vocabulary Instruction: A Conceptual Framework for Malaysian Deaf Education

Vocabulary knowledge is the gateway to literacy for Deaf and Hard of Hearing (DHH) learners, yet the incidental exposure through which hearing children acquire words is largely closed to them (Convertino et al., 2014). This conceptual paper repositions Subtitles for the Deaf and Hard of Hearing (SDH) from an accessibility service into a deliberately designed instructional resource, a linguistic catalyst for vocabulary learning (Liu, 2025), and advances a testable framework for Malaysian special primary education. The framework assigns each of three cognitive theories a distinct explanatory role: Cognitive Load Theory (CLT) defines the boundary condition imposed by a single visual processing channel (Sweller et al., 2019). The Cognitive Theory of Multimedia Learning (CTML) specifies the selection–organisation–integration sequence that SDH materials must support (Skulmowski & Xu, 2021) and Dual Coding Theory (DCT) supplies the mnemonic mechanism by which a written word form and its visual referent jointly stabilise a lexical representation (Sadoski, 2005). ADDIE provides the instructional-design framework, operationalised through two DHH-specific parameters: caption rate calibrated to learners' reading speed (Szarkowska & Gerber-Morón, 2018) and lexical complexity matched to their reading level (Zárate, 2020). Teacher readiness is hypothesised to moderate the delivery of the intervention (Zirzow, 2019). From these relationships, two falsifiable propositions are derived, providing a basis for empirically examining the proposed relationships through quasi-experimental classroom research

SDH, Deaf learners, vocabulary learning

1. Almalhy, K. M. (2022). Effect of video tutorial delivery method on D/HH students’ content comprehension. Frontiers in Psychology, 13, 872946. https://doi.org/10.3389/fpsyg.2022.872946 [Google Scholar] [Crossref]

2. Alodwan, T. A. A., & Almosa, M. (2018). The Effect of a Computer Program Based on Analysis, Design, Development, Implementation and Evaluation (ADDIE) in Improving Ninth Graders’ Listening and Reading Comprehension Skills in English in Jordan. English Language Teaching, 11(4), 43. https://doi.org/10.5539/elt.v11n4p43 [Google Scholar] [Crossref]

3. Alqraini, F. M., & Paul, P. V. (2020). The Effects of a Vocabulary Intervention on Teaching Multiple-Meaning Words to Students Who Are d/Deaf and Hard of Hearing. The Journal of Deaf Studies and Deaf Education, 25(4), 469–489. https://doi.org/10.1093/deafed/enaa015 [Google Scholar] [Crossref]

4. Baker, R. G., & Damper, R. I. (1986). Educational aspects of television subtitling in deaf education† Behaviour and Information Technology, 5(3), 227–236. https://doi.org/10.1080/01449298608914516 [Google Scholar] [Crossref]

5. Baldwin, S., Ching, Y., & Friesen, N. (2018). Online Course Design and Development among College and University Instructors: An Analysis using Grounded Theory. Online Learning, 22(2). https://doi.org/10.24059/olj.v22i2.1212 [Google Scholar] [Crossref]

6. Baranowska, K. (2019). Learning most with least effort: subtitles and cognitive load. ELT Journal, 74(2), 105–115. https://doi.org/10.1093/elt/ccz060 [Google Scholar] [Crossref]

7. Beal, J. S., Dostal, H. M., & Easterbrooks, S. R. (2024). Vocabulary (pp. 83–122). https://doi.org/10.1093/oso/9780198879114.003.0003 [Google Scholar] [Crossref]

8. Burnham, D., Leigh, G., Noble, W., Jones, C., Tyler, M. D., Grebennikov, L., & Varley, A. (2008). Parameters in Television Captioning for Deaf and Hard-of-Hearing Adults: Effects of Caption Rate Versus Text Reduction on Comprehension. The Journal of Deaf Studies and Deaf Education, 13(3), 391–404. https://doi.org/10.1093/deafed/enn003 [Google Scholar] [Crossref]

9. Cannon, J., Fredrick, L. D., & Easterbrooks, S. R. (2009). Vocabulary Instruction Through Books Read in American Sign Language for English-Language Learners With Hearing Loss. Communication Disorders Quarterly, 31(2), 98–112. https://doi.org/10.1177/1525740109332832 [Google Scholar] [Crossref]

10. Chaume, F. (2018). An overview of audiovisual translation: Four methodological turns in a mature discipline. Journal of Audiovisual Translation, 1(1), 40–63. https://doi.org/10.47476/jat.v1i1.43 [Google Scholar] [Crossref]

11. Convertino, C., Borgna, G., Marschark, M., & Durkin, A. (2014). Word and World Knowledge Among Deaf Learners With and Without Cochlear Implants. The Journal of Deaf Studies and Deaf Education, 19(4), 471–483. https://doi.org/10.1093/deafed/enu024 [Google Scholar] [Crossref]

12. Cranmer, S. (2019). Disabled children’s evolving digital use practices to support formal learning. A missed opportunity for inclusion. British Journal of Educational Technology, 51(2), 315–330. https://doi.org/10.1111/bjet.12827 [Google Scholar] [Crossref]

13. Dillon, M. R., Bolyen, E., Adamov, A. K., Belk, A. D., Borsom, E. M., Burcham, Z. M., Debelius, J. W., Deel, H. L., Emmons, A. L., Estaki, M., Herman, C., Keefe, C. R., Morton, J., Oliveira, R., Meador, A. J. S., Simard, A., Vázquez‐Baeza, Y., Ziemski, M., Miwa, H. E., … Caporaso, J. G. (2021). Experiences and lessons learned from two virtual, hands-on microbiome bioinformatics workshops. PLoS Computational Biology, 17(6). https://doi.org/10.1371/journal.pcbi.1009056 [Google Scholar] [Crossref]

14. Draxler, F., Weingärtner, H., Windl, M., Schmidt, A., & Chuang, L. L. (2023). Useful but Distracting: Keyword Highlights and Time-Synchronization in Captions for Language Learning. In arXiv (Cornell University). Cornell University. https://doi.org/10.48550/arxiv.2307.05870 [Google Scholar] [Crossref]

15. Fernández‐Batanero, J. M., Rueda, M. M., & Cerero, J. F. (2022). Are primary education teachers trained for the use of the technology with disabled students? Research and Practice in Technology Enhanced Learning, 17(1). https://doi.org/10.1186/s41039-022-00195-x [Google Scholar] [Crossref]

16. Fernández-Costales, A. (2020). Subtitling and dubbing as teaching resources in CLIL in Primary Education: The teachers’ perspective. Porta Linguarum Revista Interuniversitaria de Didáctica de Las Lenguas Extranjeras, 36, 175–192. https://doi.org/10.30827/portalin.v0i36.16228 [Google Scholar] [Crossref]

17. Fox, W. (2018). Can Integrated Titles Improve The Viewing Experience? Investigating The Impact Of Subtitling On The Reception And Enjoyment Of Film Using Eye Tracking And Questionnaire Data. In BiblioBoard Library Catalog (Open Research Library). https://doi.org/10.17169/refubium-25126 [Google Scholar] [Crossref]

18. Gernsbacher, M. A. (2015). Video Captions Benefit Everyone. Policy Insights from the Behavioral and Brain Sciences, 2(1), 195–202. https://doi.org/10.1177/2372732215602130 [Google Scholar] [Crossref]

19. Hofstätter, S., Althammer, S., Sertkan, M., & Hanbury, A. (2022). A Time-Optimized Content Creation Workflow for Remote Teaching. Proceedings of the 53rd ACM Technical Symposium on Computer Science Education, 731–737. https://doi.org/10.1145/3478431.3499421 [Google Scholar] [Crossref]

20. Holt, J. A. (1993). Stanford Achievement Test—8th Edition: Reading Comprehension Subgroup Results. American Annals of the Deaf, 138(2), 172–175. https://doi.org/10.1353/aad.2012.0684 [Google Scholar] [Crossref]

21. Kafle, S., Dingman, B., & Huenerfauth, M. (2021). Deaf and Hard-of-hearing Users Evaluating Designs for Highlighting Key Words in Educational Lecture Videos. ACM Transactions on Accessible Computing, 14(4), 1–24. https://doi.org/10.1145/3470651 [Google Scholar] [Crossref]

22. Kamaluddin, A. A., & Rashid, S. M. M. (2024). Use of Malaysian Sign Language for Special Education Teachers in Teaching Deaf Students. International Journal of Academic Research in Business and Social Sciences, 14(8). https://doi.org/10.6007/ijarbss/v14-i8/22284 [Google Scholar] [Crossref]

23. Kamarudin, D., Hussain, Y. & Darun, M. R. (n.d.). Bahasa Melayu Kod Tangan (BMKT) and Bahasa Isyarat Malaysia (BIM): Two Different Hand Signal Codes for the Hearing Impaired in Malaysian Education. https://umpir.ump.edu.my/id/eprint/22639/ [Google Scholar] [Crossref]

24. LEE, M., & Révész, A. (2018). Promoting Grammatical Development Through Textually Enhanced Captions: An Eye‐Tracking Study . Modern Language Journal, 102(3), 557–577. https://doi.org/10.1111/modl.12503 [Google Scholar] [Crossref]

25. Lewis, M. S. J. (2001). Television Literacy: Comprehension of Program Content Using Closed Captions for the Deaf. The Journal of Deaf Studies and Deaf Education, 6(1), 43–53. https://doi.org/10.1093/deafed/6.1.43 [Google Scholar] [Crossref]

26. Liu, Y. (2025). Bridging screens and classrooms: a scoping review of the pedagogical use of audiovisual translation in language education. Humanities and Social Sciences Communications, 12(1). https://doi.org/10.1057/s41599-025-05913-6 [Google Scholar] [Crossref]

27. Lo, S. (2024). Vocabulary learning through viewing dual-subtitled videos: Immediate repetition versus spaced repetition as an enhancement strategy. ReCALL, 36(2), 152–167. https://doi.org/10.1017/s0958344024000053 [Google Scholar] [Crossref]

28. Luft, P. (2017). Reading Comprehension and Phonics Research: Review of Correlational Analyses with Deaf and Hard-of-Hearing Students. The Journal of Deaf Studies and Deaf Education, 23(2), 148–163. https://doi.org/10.1093/deafed/enx057 [Google Scholar] [Crossref]

29. Lwo, L., & Lin, M. C.-T. (2012). The effects of captions in teenagers’ multimedia L2 learning. ReCALL, 24(2), 188–208. https://doi.org/10.1017/s0958344012000067 [Google Scholar] [Crossref]

30. Marschark, M., Pelz, J. B., Convertino, C., Sapere, P., Arndt, M. E. & Seewagen, R. (2005). Classroom Interpreting and Visual Information Processing in Mainstream Education for Deaf Students: Live or Memorex?. American Educational Research Journal, 42(4), pp. 727-761. https://doi.org/10.3102/00028312042004727 [Google Scholar] [Crossref]

31. May, L., Williams, A., Hassan, S., Cartwright, M., & Lee, S. (2024). Towards a Rich Format for Closed-Captioning. 1–5. https://doi.org/10.1145/3663548.3688504 [Google Scholar] [Crossref]

32. McCarron, L. (2021). Creating Accessible Videos: Captions and Transcripts. Communications of the Association for Information Systems, 48, 140–148. https://doi.org/10.17705/1cais.04819 [Google Scholar] [Crossref]

33. Moradi, S., Lidestam, B., Danielsson, H., Ng, E. H., & Rönnberg, J. (2017). Visual Cues Contribute Differentially to Audiovisual Perception of Consonants and Vowels in Improving Recognition and Reducing Cognitive Demands in Listeners With Hearing Impairment Using Hearing Aids. Journal of Speech, 60(9), pp. 2687-2703. https://doi.org/10.1044/2016_JSLHR-H-16-0160 [Google Scholar] [Crossref]

34. Nasir, N. A. M., Hashim, H., Rashid, S. M. M., & Yunus, M. M. (2021). Exploring the Potential Usage of Mobile Technologies Among the Hearing-Impaired Students in Learning English as a Second Language (ESL). International Journal of Interactive Mobile Technologies (iJIM), 15(19), 48. https://doi.org/10.3991/ijim.v15i19.22137 [Google Scholar] [Crossref]

35. Nissen, J. M., Talbot, R. M., Thompson, A. N., & Dusen, B. V. (2018). Comparison of normalized gain and Cohen’sdfor analyzing gains on concept inventories. Physical Review Physics Education Research, 14(1). https://doi.org/10.1103/physrevphyseducres.14.010115 [Google Scholar] [Crossref]

36. Paas, F., Gog, T. van, & Sweller, J. (2010). Cognitive Load Theory: New Conceptualizations, Specifications, and Integrated Research Perspectives. Educational Psychology Review, 22(2), 115–121. https://doi.org/10.1007/s10648-010-9133-8 [Google Scholar] [Crossref]

37. Parton, B. S. (2016). Video Captions for Online Courses: Do YouTube’s Auto-generated Captions Meet Deaf Students’ Needs? Journal of Open Flexible and Distance Learning, 20(1), 8–18. https://doi.org/10.61468/jofdl.v20i1.255 [Google Scholar] [Crossref]

38. Pavani, F., & Leonetti, V. (2026). On the Coexistence of Captions and Sign Language as Accessibility Solutions in Educational Settings. Audiology Research, 16(1), 20. https://doi.org/10.3390/audiolres16010020 [Google Scholar] [Crossref]

39. Perez, M. M., Peters, E., Clarebout, G., & Desmet, P. (2014). Effects of captioning on video comprehension and incidental vocabulary learning. Language Learning & Technology, 18(1), 118–141. https://doi.org/10.64152/10125/44357 [Google Scholar] [Crossref]

40. Perez, M. M., Peters, E., & Desmet, P. (2015). Enhancing Vocabulary Learning Through Captioned Video: An Eye‐Tracking Study. Modern Language Journal, 99(2), 308–328. https://doi.org/10.1111/modl.12215 [Google Scholar] [Crossref]

41. Perez, M. M., Peters, E., & Desmet, P. (2017). Vocabulary learning through viewing video: the effect of two enhancement techniques. Computer Assisted Language Learning, 31, 1–26. https://doi.org/10.1080/09588221.2017.1375960 [Google Scholar] [Crossref]

42. Sadoski, M. (2005). A Dual Coding View of Vocabulary Learning. Reading & Writing Quarterly, 21(3), 221–238. https://doi.org/10.1080/10573560590949359 [Google Scholar] [Crossref]

43. Shakeel, S. I., Mamun, M. A. A., & Haolader, M. F. A. (2022). Instructional design with ADDIE and rapid prototyping for blended learning: validation and its acceptance in the context of TVET Bangladesh. Education and Information Technologies, 28(6), 7601–7630. https://doi.org/10.1007/s10639-022-11471-0 [Google Scholar] [Crossref]

44. Si, C., Cheng, H., Su, S., Lu, M., Masud, S., Wang, Q., & Huang, Y. (2024). Motion Design Principles for Accessible Video-based Learning: Addressing Cognitive Challenges for Deaf and Hard of Hearing Learners. In arXiv (Cornell University). Cornell University. https://doi.org/10.48550/arxiv.2410.00196 [Google Scholar] [Crossref]

45. Skulmowski, A., & Xu, K. M. (2021). Understanding Cognitive Load in Digital and Online Learning: a New Perspective on Extraneous Cognitive Load. Educational Psychology Review, 34(1), 171–196. https://doi.org/10.1007/s10648-021-09624-7 [Google Scholar] [Crossref]

46. Smith, C., Crocker, S., & Allman, T. (2017). Reading Between the Lines: Accessing Information via YouTube’s Autocaptioning. Online Learning, 21(1). https://doi.org/10.24059/olj.v21i1.823 [Google Scholar] [Crossref]

47. Supermaniam, M., & Muniandy, M. N. (2025). READING IN SILENCE: EXPLORING ENGLISH READING DIFFICULTIES AND STRATEGY USE AMONG HEARING-IMPAIRED FORM 1 STUDENTS. International Journal of Modern Education, 7(25), 602–619. https://doi.org/10.35631/ijmoe.725041 [Google Scholar] [Crossref]

48. Sweller, J., Merriënboer, J. J. G. van, & Paas, F. (2019). Cognitive Architecture and Instructional Design: 20 Years Later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5 [Google Scholar] [Crossref]

49. Szarkowska, A., & Gerber-Morón, O. (2018). Viewers can keep up with fast subtitles: Evidence from eye movements. PLoS ONE, 13(6). https://doi.org/10.1371/journal.pone.0199331 [Google Scholar] [Crossref]

50. Szarkowska, A., Krejtz, I., Pilipczuk, O., Dutka, Ł., & Kruger, J. (2016). The effects of text editing and subtitle presentation rate on the comprehension and reading patterns of interlingual and intralingual subtitles among deaf, hard of hearing and hearing viewers. Across Languages and Cultures, 17(2), 183–204. https://doi.org/10.1556/084.2016.17.2.3 [Google Scholar] [Crossref]

51. Tahir, Z., Yusoff, J. Z., Thambapillay, S. & Rahman, A. S. (2021). The Concept of Equality and the Rights of People with Disabilities under the Persons with Disabilities Act 2008. Kanun: Jurnal Undang-undang Malaysia. https://doi.org/10.37052/kanun.35(2)no2 [Google Scholar] [Crossref]

52. Talaván, N. (2006). Using subtitles to enhance foreign language learning. Porta Linguarum Revista Interuniversitaria de Didáctica de las Lenguas Extranjeras. https://doi.org/10.30827/digibug.30659 [Google Scholar] [Crossref]

53. Talaván, N. (2019). Creative Audiovisual Translation Applied to Foreign Language Education. Journal of Audiovisual Translation, 2(1), 53–74. https://doi.org/10.47476/jat.v2i1.57 [Google Scholar] [Crossref]

54. Talaván, N., & Rodríguez, A. J. T. (2023). mirada transdisciplinar a la Traducción Audiovisual Didáctica. Hikma, 22(1), 143–166. https://doi.org/10.21071/hikma.v22i1.14593 [Google Scholar] [Crossref]

55. Tyler, M. D., Jones, C., Grebennikov, L., Leigh, G., Noble, W., & Burnham, D. (2009). Effect of caption rate on the comprehension of educational television programmes by deaf school students. Deafness & Education International, 11(3), 152–162. https://doi.org/10.1002/dei.262 [Google Scholar] [Crossref]

56. ÜNLÜER, S. (2011). BARRIERS IN THE INTEGRATION OF ICT AT THE SCHOOL FOR THE HANDICAPPED. Problems of Education in the 21st Century, 33(1), 92–110. https://doi.org/10.33225/pec/11.33.92 [Google Scholar] [Crossref]

57. Yoon, J.-O., & Kim, M. (2011). The Effects of Captions on Deaf Students’ Content Comprehension, Cognitive Load, and Motivation in Online Learning. American Annals of the Deaf, 156(3), 283–289. https://doi.org/10.1353/aad.2011.0026 [Google Scholar] [Crossref]

58. Zárate, S. (2020). Captioning and Subtitling for d/Deaf and Hard of Hearing Audiences. In UCL Press eBooks. UCL Press. https://doi.org/10.2307/j.ctv14t478b [Google Scholar] [Crossref]

59. Zirzow, N. K. (2019). TECHNOLOGY USE BY TEACHERS OF DEAF AND HARD-OF-HEARING STUDENTS. https://doi.org/10.33915/etd.3885 [Google Scholar] [Crossref]

• The Impact of Ownership Structure on Dividend Payout Policy of Listed Plantation Companies in Sri Lanka
• Urban Sustainability in North-East India: A Study through the lens of NER-SDG index
• Performance Assessment of Predictive Forecasting Techniques for Enhancing Hospital Supply Chain Efficiency in Healthcare Logistics
• The Fractured Self in Julian Barnes' Postmodern Fiction: Identity Crisis and Deflation in Metroland and the Sense of an Ending
• Impact of Flood on the Employment, Labour Productivity and Migration of Agricultural Labour in North Bihar