Integrating AR and AI in Mandarin Language Education: An Analysis

Integrating AR and AI in Mandarin Language Education: An Analysis

Goh Chin Shuang^1*, Hao Yixuan²,  Airil Haimi Mohd Adnan³,  Zhang Huiqin⁴, Peng Jie⁵ 

^1,3,4Akademi Pengajian Bahasa, Universiti Teknologi MARA, 40450, Shah Alam, Malaysia 

²Kunming University of Science and Technology, People’s Republic of China 

⁵Queen Mary University of London, Mile End Road, Bethnal Green, London E14NS, UK 

*corresponding author 

DOI: https://dx.doi.org/10.47772/IJRISS.2024.8080223

Received: 13 August 2024; Accepted: 21 August 2024; Published: 13 September 2024

ABSTRACT

This study examines the views of Malaysian undergraduates who were exposed to Augmented reality (AR)and Artificial Intelligence (AI) technologies to learn Mandarin Chinese language as non-native speakers. The primary aim is to investigate the learning needs of these language learners as they are progressively exposed to an AR-AI learning environment, together with their preferred contents and design. This is a fully quantitative study employing an elaborate Cognitive Theory of Multimedia Learning survey questionnaire; the respondents included 583 undergraduates from different faculties at Universiti Teknologi MARA in the state of Selangor. After a month of Mandarin instruction (through Introductory Mandarin Levels I, II, or III), data were gathered and analysed using SPSS. Findings show that the measurement instrument is reliable in measuring respondents’ views on AR-AI in Mandarin learning. Respondents from amongst degree students in Science and Technology, Business, Social Sciences and Humanities were surveyed and their general opinion regarding the course contents and presentation were largely positive. From the findings, the respondents believe that the AR-AI application they used contributed to their acquisition of beginner level Mandarin through images, text, and voice clips. In addition, these technologies promoted active learning and minimised cognitive overload, which guided the application’s original multimedia design.

Keywords: Chinese language, Augmented Reality (AR), Artificial Intelligence (AI), multimedia, instructional design, Malaysia

INTRODUCTION

These opening paragraphs aim to highlight, with particular attention to their uses in language acquisition and Mandarin Chinese instruction, the benefits of Augmented Reality (AR) and Artificial Intelligence (AI) in education. Moreover, it will draw attention to the present research vacuum on the application of these technologies for Mandarin language acquisition in the Malaysian higher education sector, thereby facilitating more study and possible deployment plans.

It goes without saying that AR and AI technologies in education have transformed conventional teaching and learning models. These innovative technologies provide interactive, personalised, and immersive experiences that improve student involvement and achievements across several fields (Radianti et al., 2020). Among the uses of AR and AI in education, language learning has become especially interesting as it has great possibility to enhance student experiences as well as to augment old teaching strategies. From increased motivation and engagement to tailored learning routes and real-time feedback, AR and AI have many benefits for education. These technologies provide opportunities in language education to build real, context-rich settings that support more natural and efficient acquisition of language skills (Shadiev et al., 2020). It is thus not surprising that researchers and practitioners have been paying more attention to AR and AI applications in language education as the worldwide need for multilingual competency keeps rising.

As one of the most common languages used worldwide and an increasingly necessary language for international business and cultural interactions, Mandarin Chinese has grown to be a focus of many creative AR and AI-based learning projects. With its tonal system and character-based writing, Mandarin’s complicated nature offers special difficulties that these technologies are well equipped to solve (see, Yang et al., 2020). From interactive character recognition to immersive conversational practice, AI and AR are quickly changing the field of Mandarin language instruction.

Driven by economic linkages to China and the multicultural character of Malaysian society, Mandarin competency has become much more important within the framework of Malaysian higher education. Though AR and AI might help Mandarin language learners, there seems to be a clear study vacuum about their uses and efficiency inside Malaysian colleges and universities (see, Chung et al., 2020). This gap offers a significant research potential as knowledge of the particular difficulties and possibilities in this context will enable more efficient and culturally relevant uses of various technologies.

The integration of AI and ARtechnologies in educational environments has shown several advantages in improving the whole learning process. The possibility to design interactive and immersive learning environments is one of the main benefits. AR overlays digital data onto the actual environment so that students may interact with instructional materials in a more physical and hands-on way (Bacca et al., 2014). At the other end of the spectrum, AI introduces to education the principle of adaptive learning. Analysing student performance and behaviour helps AI systems to customise instructional materials and pace to fit individual learner demands, therefore guaranteeing a more individualised and effective learning environment (Chen et al., 2020). Dealing with the many learning styles and speeds observed in every classroom environment requires this flexibility, which also helps to provide a more inclusive learning environment.

Furthermore, AR and AI technologies together allow for the development of so-called ‘intelligent teaching systems’that are capable of giving students quick feedback and direction. Offering students the advantages of individualised education at scale, this real-time engagement can closely replicate one-on-one coaching (Ma et al., 2014). These tools greatly improve the learning process by pointing out areas where students struggle and offering focused help. To boot, AR and AI-based learning technologies’ originality and interactivity can easily grab students’ interest and help to make the learning process fun. Increased participation in educational activities and better knowledge retention can result from this higher involvement level (Garzón et al., 2019).

Augmented Reality and Artificial Intelligence in language education

With reference to language education, AR and AI technologies present special benefits that help to solve some of the difficulties with conventional language instruction approaches. These technologies open doors for contextualised learning, immersive language experiences, and tailored instruction that can greatly improve language acquisition. Frequently, AR applications in language education centre on building real, context-rich settings for practical language use and vocabulary development. AR applications, for example, may overlay foreign language labels on actual things so that students may investigate their environment while concurrently learning new vocabulary in context (Ibrahim et al., 2018). This method not only promotes memorisation but also helps students to connect ‘fresh’ terms with their real-world referents, therefore enabling more natural language learning.

Chatbots and virtual language partners, among other AI-powered language learning technologies, let students practise conversational skills in a low-stress setting. These AI systems may interact, give pronunciation feedback, and modify their language difficulty to fit the learner’s degree of competency (Fryer et al., 2019). More frequent practice and learner confidence building are made possible by this ongoing availability of a ‘conversation partner’.Moreover, AI systems may examine student performance to find areas of challenge and customise activities to target certain shortcomings. This tailored strategy guarantees that students concentrate on the features of language they find most difficult, thereby optimising the effectiveness of their study time (Zuidema et al., 2021).

Given the innate difficulties presented by Mandarin Chinese learning, the implementation of AR and AI technologies offers very fascinating prospects. Mandarin is a perfect target for creative technology solutions as its tonal quality and character-based writing system may prove challenging for learners from non-tonal language backgrounds.Character recognition and writing exercise performance of AR applications for Mandarin learning have shown a lot of potential. AR applications can lead students through the difficult process of character creation by overlaying stroke order animations on tangible items or textbooks (Yang et al., 2021). Some uses of AR even offer interactive settings where students may experience Chinese cultural settings, therefore improving language abilities and cultural knowledge.

On the other hand, tools driven by AI for Mandarin acquisition sometimes concentrate on tonal correctness and pronunciation. Instant feedback on tonal pronunciation given by advanced voice recognition systems can help students to acquire this fundamental component of the language (Tsai, 2019). Also, AI chatbots made especially for Mandarin can interact with students in discussions that emphasise correct tonal use and common grammatical patterns.AI systems may also examine student mistakes to find trends and offer focused workouts. For example, the system may provide practice materials emphasising on certain difficulties if a student regularly faces certain issues with particular tonal combinations (Hsu et al., 2021).

Researching Artificial Intelligence and Augmented Reality in Malaysian higher education

Although AR and AI technologies clearly have great promise for Mandarin language instruction, their use and efficacy in the Malaysian higher education sector are lacking in study. With its cosmopolitan population and developing business relations with China, Malaysia offers a special setting for learning Mandarin that calls particular attention.Current empirical studies on AR and AI in language learning within Malaysia have mostly concentrated on English language acquisition, therefore leaving a vacuum on how these technologies may be deployed successfully to Mandarin education in Malaysian universities and colleges (Chung et al., 2020). Given the growing value of Mandarin competency in the Malaysian labour market and the nation’s strategic orientation in China’s Belt and Road Initiative, this disparity is especially significant.

Many elements help to explain this gap in our knowledge. First of all, not much information exists on the particular difficulties that Malaysian university students have studying Mandarin, which would guide the creation of focused AR and AI solutions. Second, given the many mother tongues and cultural backgrounds of students, Malaysia’s linguistic and cultural varieties might call for different ways of using these technologies (Tan et al., 2020).Examining technologyinfrastructure, teacher development, and student approval of AR-AI learning environments could shed important light on how best to use AR and AI to improve Mandarin language instruction in Malaysian institutions (Md Ali et al., 2020). Such studies might even influence curriculum development, advise policy choices, and eventually help Mandarin language instruction in Malaysian higher education to be more effective and of higher quality.

Based on the preceding discussion, the two objectives of this present study are to investigate students’ perception regarding the integration of AR and AI in learning Mandarin, especially for non-native speakers. Specifically, the study is conducted:

1. To investigate the learning needs and preferences of non-native Mandarin Chinese learners withinAR-AI learning environment, and,
2. To identify the salient features in the presentation of multimedia contentof an AR-AI application (app) for Mandarin Chineselanguage learning.

To reiterate, the research vacuum that has been observed with regards to the application and efficiency of AR and AI technologies for Mandarin language acquisition in the Malaysian higher education sector poses both a challenge and an opportunity. By doing focused research, we can bridge this gap and create AR and AI-based learning solutions that are culturally suitable and useful for Malaysian learners and institutions. It is essential to investigate how these advanced technologies may be effectively used to improve Mandarin language instruction in Malaysia’s varied and dynamic higher education system as we progress. This research has the capacity to enhance language learning outcomes and bolster Malaysia’s standing in a progressively interconnected global economy, where fluency in Mandarin is getting increasingly valued.

LITERATURE REVIEW

Integrating Augmented Reality and Artificial Intelligence in language learning

With the advancement of digital learning tools, Augmented Reality (AR) and Artificial Intelligence (AI) based technologies have been widely used in education in recent years. Various research studies have been conducted regarding the benefits of applying AR and AI in education.

The learning environments based on AR technologies have already been proven effective as an active learning method due to their ability to translate the learned content into long-term memory (Santos et al., 2014). AR technology can enhance the learning experience by making it more interactive and engaging, as learners can interact with virtual objects in the real world (KI Afrashtehfar et al., 2021; Zhang, 2018). Khoshnevisan and Le (2018) categorized the benefits of AR in language education into dimensions of learning outcomes and affective outcomes. Parmaxi and Demetriou (2020) systematically analyzed research published between 2014 and 2019 and identified the main benefits of AR in language learning, which included: (1) increased motivation; (2) improvement of learning outcomes; (3) enhancement of interactions; and (4) creation of opportunities for authentic language tasks.

Özcelik et al. (2022) revealed benefits of integrating AR in education such as: (i) access to AR learning materials without boundaries of space and time; (ii) affordability of AR learning materials; (iii) portability of AR learning materials; and (iv) suitability for any field of study in education. Additionally, other researchers indicate that integrating AR in learning facilitates learners (Wang, et al.,2017), promotes enhanced learning achievement (Murat & Gökçe,2017). AR in language learning promotes immersive learning (Belda-Medina & Marrahi-Gomez, 2023; Huanget al., 2021;Kazu & Kuvvetli, 2023; Mozaffari & Hamidi, 2023; Shadiev & Liang, 2023), boosts motivation (Huang et al., (2021);Chang et al., 2022; Chuang et al., 2022; Kazu & Kuvvetli, 2023; Wang et al., 2022), promotes fun learning (Kazu & Kuvvetli, 2023), increases learning retention (Belda-Medina & Marrahi-Gomez, 2023; Kazu & Kuvvetli, 2023; Mozaffari & Hamidi, 2023; Shadiev & Liang, 2023), reduces learning anxiety (Huang et al., (2021) and reduces cognitive load (Kazu & Kuvvetli, 2023).

AI in language is not new and has become a common tool in our daily lives. Smartphones, computers, and laptops are equipped with AI assistants (e.g., Siri, Google Assistant, Gemini, Copilot GPT-4). The development of Chat Generative Pretrained Transformer (ChatGPT) by OpenAI has significantly accelerated public exposure to AI (Ghumra, 2022; Lund & Wang, 2023). Kessler (2023) revealed the potential of integrating AI technology into Computer Assisted Language Learning (CALL).

Artificial Intelligence (AI) provides effective and flexible platforms for designing learning environments with crucial characteristics, leading students to success (Devitska, 2019). Borge (2016) found that AI facilitates precise student assessments for instructors, helping to identify deficiencies in lectures, scientific content, and educational materials. AI adjusts assignments, monitors individual progress, and offers feedback through intelligent programs. Additionally, AI tools effectively manage classroom density. Research by Viktorivnaet al.(2022) found that AI significantly improves the quality of language learning by adapting to each student’s individual features (talent and background) and expectations (aims and objectives). Chatbots, a common AI-based technology, are very useful for engaging students in lessons, especially in practice activities through mobile-based applications and social networks (Nghi et al., 2019).

In language learning, Soomro et al. (2023) found that AI positively influences English Language Learners (ELLs) by streamlining the learning procedures for listening, speaking, reading, and writing skills. AI can also be used to experimentally prove linguistic theories and understand aspects of human cognition regarding language learning and acquisition (Almelhes, 2023). Khan (2022) pointed out that AI technological advancements go beyond traditional methods and frameworks of language teaching and learning used in applied linguistics, making educational linguistics an interesting and unpredictable area of study.

Pokrivakova (2019) stated that applying AI in foreign language education provides learners with immediate and highly individualized support. AI-powered tools can continually analyze each student’s outputs, diagnose their individual learning needs, adapt the learning content accordingly, and give well-grounded feedback within seconds. These tools can collect massive amounts of data on students’ learning progress, model their personal learning curves, and adapt learning content accordingly. They enhance learners’ progress through small consequential steps and immediate feedback. This quick feedback assists language teachers in monitoring and identifying their students’ progress effectively, enabling early intervention (Khalilova et al., 2024). AI in language learning also frees teachers from tiring and time-consuming activities such as grammar or pronunciation drills (Pokrivakova, 2019).

Zuriana (2020) found that AI systems can understand natural human speech and incorporate a flipped learning approach for language teaching and learning, making learners more competent and productive while assessing their speech. Lee et al. (2024) and Hlongwane et al. (2024) demonstrated that AI supports individual language learning opportunities by assessing progress and proposing efficient individual instruction, proving useful for language learning development. AI frequently assists students in learning writing (Godwin-Jones et al., 2024), reading, vocabulary, grammar, speaking, and listening. Natural language processing, automated speech recognition, and learner profiling are commonly applied to develop automated writing evaluation, personalized learning, and intelligent tutoring systems (Huang et al., 2023).

Mohamad (2024) depicted those technologies such as AR, AI, and the metaverse can tailor language classes to fit each learner’s unique style and pace. The frontiers of augmented reality (AR), artificial intelligence (AI), and the metaverse offer rich educational possibilities. These tools are being used in Chinese language classes to enhance learners’ linguistic efficiency and knowledge of China’s history and culture.

Research on Designing, Presentation, and Content of AI and AR in Language Learning

Recent research has explored various aspects of design, presentation, and content in AR and AI applications for language learning. Studies have investigated the effectiveness of different instructional strategies, interface designs, and interaction modalities in AR-enhanced language learning environments (Billinghurst et al., 2015).

A study by Fan et al. (2020) and Goh et al. (2023)found that vocabulary learning is one of the most suitable components of language learning to master with AR. Activities such as word spelling games, word knowledge activities, and location-based word activities were particularly effective. The AR material designed in this study aimed to recommend new vocabulary words to primary-level students, incorporating animation and sound to make the material more engaging (MHM Jamrus & AB Razali, 2019).

Kondo’s research (2006) showed that AR is advantageous in expanding the functionalities of traditional educational textbooks. This was demonstrated with educational material developed to explain the structure of the human brain using 3DCG (computer-generated 3D technology) and sound technology. Miyosawa et al. (2012) stated that the creation of AR-based educational materials includes a few steps: (1) development environment; (2) application of AR; (3) deciding which foreign language to use; (4) development of teaching materials; (5) verification test. AR applications can be categorized into three types: image-based, creation-based, and markerless AR (Karacan & Akoglu, 2021).

IT-powered tools can be applied in many ways. Pokrivcakova (2019) identified eight applications: (1) personalized learning materials; (2) machine translation tools; (3) AI writing assistants; (4) chatbots; (5) AI-powered language learning software and apps; (6) intelligent tutoring systems; (7) adaptive and intelligent systems for collaborative learning support; and (8) intelligent virtual reality.

Individual differences are critical for developing pedagogical tools to target specific students and tailor education to individual needs at different stages. Intelligent educational systems employing big data and AI techniques can collect accurate and rich personal data (Luan et al., 2020). Education is progressively moving from a one-size-fits-all approach to precision education or personalized learning (Hlongwane et al., 2024; Lee et al., 2024; Lu et al., 2018; Tsai et al., 2020). Generative AI tools have the potential to offer new and innovative ways to support both teaching and self-directed learning. AI image generator tools such as DALL-E3 can help teachers instantly create relevant images for use in lessons. Generative AI chatbots such as ChatGPT can generate tailored lesson plans, create personalized and graded texts, construct writing templates, and provide scaffolded language exercises and activities (Kohnke et al., 2023; Lee, Shin et al., 2023). It is important for teachers to have the competencies needed to capitalize on the affordances of generative AI tools while being critically aware of their drawbacks and limitations. An important role for teachers will be to guide students in using generative AI tools to support their learning, rather than completing assessed assignment tasks on their behalf (Yeo, 2023).

Woo and Choi (2021) consolidated data on AI tools created from 2017 to 2020. Most of these tools employed machine learning and natural language processing, aiming to detect errors, offer feedback, and evaluate language proficiency. Educational AI tools enhance learner engagement and motivation (Divekar et al., 2021). For example, a narrative-based learning system equipped with chatbot feedback significantly improved user engagement (Ruan et al., 2020).

Likewise, a mobile chatbot-based learning approach enabled nursing students to believe that chatbots could promote their learning engagement (Chang et al., 2022). The main components of the chatbot application include interactions, connections, chat bubbles, text-to-speech, prior responses, quick replies, custom variables, arithmetic operations, embedding media, and connecting channels (Mageira et al., 2022).

Diverkar et al. (2021) integrated AI and XR (extended reality) technology tools into learning Mandarin as a foreign language. They developed a learning paradigm called the Cognitive Immersive Language Learning Environment (CILLE) within the AI+XR environment. This paradigm utilized panoramic scenes and real-world 360° images to create a virtual environment where learners could acquire vocabulary, cultural knowledge, practice sentence-level exercises, engage in listening activities, and participate in conversations with multimedia elements such as audio, video, images, and text.

The pinyin is associate with Chinese character and translation to let student easy to under the meaning of the content.  Students participated in role-play activities within true-to-life locations, enabling them to immerse themselves in Mandarin learning and practice. The design also incorporated gamification elements to create a fun and enjoyable learning environment.

The research revealed significant improvements in students’ proficiency in vocabulary, listening, comprehension, and conversation. Additionally, there were notable enhancements in students’ willingness to communicate in the target language, cultural absorption, incidental learning, and the provision of motivation, instruction, and feedback for language learning.

Khalilova et al. (2024) indicated that advances in technology tools such as AR and AI offer multimedia materials embedded with elements like video and audio, exposing students to language use in real-world situations. By exploring resources such as podcasts and online articles, students can accommodate different learning styles and develop individual cultural awareness.

METHODOLOGY

Method and Instrument

The research utilized a quantitative approach (Creswell,2003; MertlerandCharles,2014), employing an online survey questionnaire created with Google Forms. The questionnaire comprised three sections; Section A (demographics), Section B (presentation of Mandarin course content), and Section C (presentation of multimedia content for learning Mandarin). The items in Section C were rated rest on a five-point Likert scale ranging from 1 to 5: 1=Strongly Disagree (SD), 2=Disagree (D), 3= Uncertainty (U), 4=Agree (A), and 5=Strongly Agree (SA).

Sampling

The sample for this study was chosen using purposive sampling, comprising students from various faculties at Universiti Teknologi MARA (UiTM) Selangor who were enrolled in Mandarin classes.

In total, 583 undergraduate students volunteered to participate in the study. These students were non-native speakers of Chinese/Mandarin and were enrolled in Introductory Mandarin levels I, II, and III at UiTM, Selangor. They were drawn from three main categories: Science and Technology, Business, and Social Sciences & Humanities. These faculties are located at both the UiTM Shah Alam main campus and the satellite campus in Puncak Alam.

The faculties within the Science and Technology category include the Faculty of Built Environment (AP), the Faculty of Applied Science (AS), the Faculty of Computer and Mathematical Sciences (CS), the Faculty of Civil Engineering (EC), the Faculty of Electrical Engineering (EE), the Faculty of Mechanical Engineering (EM), the Faculty of Health Science (HS), the Faculty of Pharmacy (PH), and the Faculty of Sports Science & Recreation (FSR). The Business category encompasses the Faculty of Accountancy (AC), the Faculty of Business Management (BM), and the Faculty of Hotel & Tourist Management (HM).Lastly, the Social Sciences & Humanities category comprises the Faculty of Art and Design (AD), the Faculty of Education (ED), the Faculty of Performing Arts (FF), the Faculty of Administrative Science & Policy Studies (IM), the Faculty of Law (LW), the Faculty of Mass Communication & media studies(MC), and the Faculty of Music (MU).

Table 1: Sample distribution

The sample was contributed from three categories: Science and Technology, Business, and Social Sciences & Humanities. Among them, the Science & Technology faculty (AP, AS, CS, EC, EE, EM, HS, PH, FSR) consists of 302 students, accounting for 51.8% of the total sample.

Following this, Social Sciences & Humanities (contributed AD, ED, FF, IM, LW, MC, MU) 167 students, representing 28.6%, while Business (AC, BM, HM) had only 114 students, making up 19.6%. This distribution indicates that our sample covers a diverse range of fields of study, assuming that the Mandarin proficiency level among them is relatively similar.

Upon referencing the table 1, it becomes apparent that females outnumber males significantly, with 470 female students (80.6%) compared to only 113 male students (19.4%). This gender imbalance is a common trend at UiTM, where female enrolment tends to exceed that of males in various courses.

We categorized the participants into three Mandarin proficiency levels to gather perceptions on integrating ARAI technology into Mandarin learning. Mandarin Level I participants were the most numerous, comprising 282 students (48.4%), while Levels II and III were nearly equal, with 150 and 151 students, respectively (25.9% and 25.7%). The age range of students enrolled in Mandarin courses spans from 18 to 24, with the majority falling between the ages of 20 and 21 (62.4%).

Data Collection and Analysis

The data were collected after the students had been enrolled in the Mandarin class for one month. Subsequently, the questionnaire, in the form of a Google Form, was distributed to students via WhatsApp with the assistance of lecturers. Students who were willing to participate in the study were invited to complete the form, which remained open for two weeks.

The collected data were analyzed using the Statistical Package for the Social Sciences (SPSS) to assess the reliability of questionnaire items and to calculate descriptive statistics such as frequency, percentage, mean scores, and standard deviation. The results were then presented in tables.

Reliability of Questionnaire Items

Table 2 presents the Cronbach’s Alpha coefficients indicating the internal consistency of participants’ perceptions towards integrating ARAI (Augmented Reality and Artificial Intelligence) in learning Mandarin, specifically focusing on the presentation of course contents and multimedia materials. The Cronbach’s Alpha for the presentation of course contents of ARAI was calculated to be 0.987, while for the presentation of multimedia contents, it was 0.985. The overall Cronbach’s Alpha coefficient for this perception study was determined to be 0.986. These findings align with Gliem & Gliem’s (2003) suggestion that coefficients exceeding 0.9 represent excellent reliability, indicating that the instrument possesses acceptable internal consistency. Therefore, the results imply a high level of reliability in measuring participants’ perceptions towards integrating ARAI in Mandarin learning, affirming the robustness of the employed measurement instrument.

Table 2: Constructs and Items

FINDINGS

In this study, the adequacy of the dataset for factor analysis was assessed using the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity. The KMO statistic evaluates the proportion of variance among variables that might be common variance, with values closer to 1 indicating better suitability for factor analysis. Our analysis yielded a KMO value of 0.979, significantly exceeding the recommended threshold of 0.6, thereby affirming the high adequacy of our dataset for factor analysis (Kaiser, 1974). Bartlett’s test of sphericity further validated these findings, yielding an approximate chi-square value of 25181.157 with 406 degrees of freedom, and a significant p-value of 0.000 (p < 0.05). This significant result indicates that the correlations among variables are sufficiently large to justify the use of factor analysis (Bartlett, 1954).” In other words, since the KMO sampling suitability measure was greater than 0.6, and the significance was <0.05, so the validity of the questionnaire was passed.

Table 3: KMO and Bartlett test

Findings related to the first research objective (learning needs and preferences of non-native Mandarin Chinese learners within an AR-AI learning environment)

Table 4 revealed students’ perceptions of the presentation of course content in Mandarin AR AI. Most students gave positive feedback regarding this. The presentation of the course contents is divided into:

i) The design part: instruction, interface layout, navigation menu, course banner, and user log/record. ii) The content part: course information, learning outcomes, subtopics of content, exercises/games/quizzes, and the use of languages and translation. iii) The learning activities: forum, discussion activities, and comments.

For the design part of Mandarin ARAI contents, 467 students (80.10%) agreed that the instruction should be easy to understand. 475 students (81.47%) agreed that the interface design should be attractive. The navigation menu of course contents can help learners access the contents of ‘Mandarin ARAI’ easily, with 481 students (82.50%) agreeing. 470 students (81.61%) also agreed that the course banner and user log/record of “Mandarin ARAI” should be well-organized, with 479 students (82.16%) concurring.

Regarding the presentation of “Mandarin ARAI” course content, most students (490 students; 84.05%) agreed that the course information should be clearly stated. The learning outcomes of each topic should be clearly understood, and the subtopics of each topic area should be divided in an orderly manner, with 480 students (82.34%) and 482 students (82.68%) respectively agreeing.

Additionally, 439 students (83.87%) agreed that various teaching materials (videos, notes, etc.) must facilitate learners in better understanding the “Mandarin ARAI” lesson. 487 students (83.53%) responded that the exercises/games/quizzes are effective, and 493 students (84.49%) reflected that the use of languages and translation should be well understood.

According to the learning activities such as forums and discussion activities, more than 80% of students responded that such activities are important in learning Mandarin. 471 students (81.82%) agreed that the forum provided should help learners interact with other students.

Additionally, 476 students (81.65%) agreed that the discussion activities in the forum should encourage learners to share their understanding with friends/peers and that the comment space should help learners interact with the facilitators. In other words, interaction and participation in forums or discussion activities among peers or facilitators are important in learning Mandarin.

Even though more than 80% of students provided positive feedback about the presentation of course content, nearly less than 5% of students did not agree with the design and presentation of the Mandarin ARAI contents.

Table 4: Student’s perception on the presentation of course content

Referring to the data collected, the mean of the results ranges from 4.11 to 4.20 (Gliem & Gliem, 2003), with a standard deviation of 0.802-0.863. This indicates that the presentation of course content is important for developing Mandarin AR AI materials. The highest mean scores were recorded by three items: “Various teaching materials (videos, notes, etc.) must facilitate learners in better understanding the lesson.” (M = 4.20, SD = 0.831), “The use of language in instructions should be well understood.” (M = 4.19, SD = 0.831), and “Translations of texts into Bahasa Malaysia must facilitate students in understanding the contents.” (M = 4.19, SD = 0.839). This means that teaching materials and language play an important role in designing “Mandarin ARAI” apps for learning Mandarin. Thus, it can be concluded that students are positive about integrating Augmented Reality (AR) and Artificial Intelligence (AI) technology in designing Mandarin digital content.

Findings related to the second research objective (salient features in the presentation of multimedia content of an AR-AI app for Mandarin language learning)

Questions 17 to 25 assess students’ perceptions of multimedia content presentation in the development of the “Mandarin ARAI” application for learning Mandarin. Respondents were asked to indicate their agreement on the suitability of various multimedia elements within the app, including text readability, virtual graphics/images, audio quality, video quality, and overall interface design. Based on descriptive survey data analysis, a majority of the 538 students provided positive feedback.

504 students (86.45%) demonstrated that the text in the “Mandarin ARAI” app should be easy to read and suitable. Regarding virtual graphics/images, students emphasized that they should be attractive and appropriate, aiding learners in better understanding lessons. This suggests that appealing and suitable images are crucial and can motivate students to engage with Mandarin learning. In terms of audio, 508 students (87.13%) highlighted the importance of high-quality audio for the design of Mandarin ARAI.

In terms of layout and interface design, 504 students (86.45%) agreed that it should be systematic and user-friendly. Regarding navigation, 501 students (85.93%) emphasized its importance in the design and presentation elements of the Mandarin ARAI app. Regarding animated video content, 82.34% of students responded positively regarding high-quality animation, with 85.76% indicating that animations should enhance understanding of Mandarin learning, making it more enjoyable (86.62%).These findings align with those of Garzón et al. (2019) and Kazu & Kuvvetli (2023), who revealed that the AR promotes fun and enjoyable learning.

Table 5: Student’s perception on the presentation of multimedia content

In addition, descriptive analysis, mean, and standard deviation from Table 5 underscore the significant role of multimedia content presentation in Mandarin ARAI application design. Overall mean scores ranged from 4.18 to 4.30, with standard deviations between 0.822 and 0.866, indicating that the Mandarin ARAI app is highly regarded as effective and suitable e-supplementary tools for learning Mandarin. Items with the highest mean values included “The audio quality should be clearly audible” (N=4.30, SD=0.839), followed by “The navigation of Mandarin ARAI should be user-friendly” and “Animated videos should make learning more fun” (both with mean=4.28, SD=0.822 and SD=0.829, respectively).

The item with the lowest mean value was “The quality of the video must be high” (M=4.12, SD=0.866). In conclusion, students welcome the integration of multimedia content in the “Mandarin ARAI” app for learning Mandarin, perceiving it as enjoyable and enhancing their understanding of the language.

CONCLUSIONS

This study investigated the learners’ feedback in relation to the presentation of the ARAI app in its design contents and Mandarin learning activities. Besides, the presentation of multimedia content of learning Mandarin has also been investigated. From the study finding, the study confirmed the potential of ARAI app as a supplementary e-instructional tool in the teaching and learning of Elementary Mandarin for UiTM students.

According to the presentation of Mandarin course content in term of design, ARAI content and the learning activities, majority students respond in positive feedback. For design part, the study confirms that, the instruction, interface layout, navigation menu, course banner and user log/record of ARAI apps should be interactive, easy to access and well organised.  Regarding ARAI content, the results from the questionnaire showed that course content should be clearly stated and incorporated with learning outcomes (Khoshnevisan & Le, 2018;Parmaxi&Demetriou, 2020). The subtopics of each topic should be arranged in order manner and teaching materials must facilitate learners better understanding “Mandarin ARAI’ lesson. The exercise/game/quiz and translation provided can assist learners better comprehend Mandarin (Diverkar et al.,2021)

From the research findings regarding the learning activities, most of the students agreed that learning activities should be assist learner can interact with peers or instructors (Parmaxi & Demetriou, 2020). Additionally, discussion activities should encourage students to share their knowledge among peers.

Referring to the data collected, in terms of ARAI multimedia presentation of content, majority of the respondents think that integrating multimedia elements (audio, text, animation, and image) in ARAI app for learning Mandarin should be systematic and user friendly (Özcelik et al.,2022; Goh et al.,2023). Besides, ARAI presentation, interface design/layout should also be systematic and use friendly too. Whereas, content presentation and  navigation for content should be designed to be easily comprehended by the users.

RECOMMENDATIONS

The results regarding the presentation of multimedia content suggest that the Mandarin ARAI app is suitable for facilitating the learning of Mandarin through appropriate use of graphics, text, and audio. These guidelines can assist practitioners, instructional designers, and researchers who intend to develop ARAI instructional materials for language learning, especially for Mandarin.The multimedia design and development of the app align with the Cognitive Theory of Multimedia Learning (Mayer, 2001; Clark & Mayer, 2003), which advocates for combining words (in the form of text), graphics, audio, and video in a manner that promotes active learning while avoiding cognitive overload. Additionally, the design and layout of the ARAI app should be systematic and user-friendly.

Further research efforts should focus on refining the design and content of the ARAI Mandarin learning materials to better meet learners’ needs and expectations. This includes ensuring that graphics, text, audio, and video are used effectively to support learning. It is also essential to ensure that the app’s interface remains user-friendly and systematic, enhancing the overall learning experience. Exploring ways to further boost learners’ motivation and engagement through the Mandarin ARAI app, potentially incorporating gamification elements or interactive features (Diverkar et al., 2021; Khalilova et al. 2024), is recommended.Furthermore, the findings from this study, which are based on the Cognitive Theory of Multimedia Learning (Mayer, 2001; Clark & Mayer, 2003), can serve as guidelines for developing future iterations of the ARAI app and similar educational tools.

To sum up, this study highlights the critical role of well-designed multimedia content in the Mandarin ARAI app and its potential to enhance learners’ motivation and understanding. Future research should continue to build upon these insights to optimize the design and development of new multimedia language learning apps.

ACKNOWLEDGEMENTS

This work was funded by Kumming University of Science and Technology, Center for Language Education and Cooperation, China, and supported by Akademi Pengajian Bahasa, Universiti Teknologi MARA, Malaysia,

REFERENCES

1. Afrashtehfar, K. I., Yang, J. W., Al-Sammarraie, A., Chen, H., & Saeed, M. H. (2021). Pre-clinical undergraduate students’ perspectives on the adoption of virtual and augmented reality to their dental learning experience: A one-group pre-and post-test design protocol. F1000Research, 10.https://doi.org/12688/f1000research.53059.2
2. Ali, Zuraina. (2020). Artificial intelligence (AI): A review of its uses in language teaching and learning. IOP Conference Series: Materials Science and Engineering. Vol. 769. No. 1. IOP Publishing.
3. Bacca, J., Baldiris, S., Fabregat, R., Graf, S., & Kinshuk. (2014). Augmented Reality Trends in Education: A Systematic Review of Research and Applications. Educational Technology & Society, 17(4), 133-149
4. Barnes-Hawkins, C. (2016). English language learners’ perspectives of the communicative language approach. Available at: https://scholarworks.waldenu.edu/cgi/viewcontent.cgi?article=3623&context=dissertations&httpsredir=1&referer=.
5. Bartlett M. S. (1954). A further note on the multiplying factors for various chi-square approximations in factor analysis. Journal of the Royal Statistical Society, Series B, 16, 296-298.
6. Billinghurst, M., Clark, A., & Lee, G. (2015). A survey of augmented reality. Foundations and Trends® in Human–Computer Interaction, 8(2-3), 73-272.
7. Borge, N. (2016). White paper – Artificial Intelligence to Improve Education/Learning Challenges. International Journal of Advanced Engineering & Innovative Technology, 2 (6), 10–13. https://doi.org/10.32595/iirjet.org/v2i2.2016.29
8. Chang, C.Y.,Hwang, G.J.,Gau, M.L. (2022).Promoting students’ learning achievement and self-efficacy: A mobile chatbot approach for nursing training. Brit. J. Educ. Technol. 53, 171–188.
9. Chang, Y. S., Chen, C. N., & Liao, C. L. (2020). Enhancing English-learning performance through a simulation classroom for EFL students using augmented reality—A junior high school case study. Applied Sciences, 10(21), 7854.
10. Chen, X., Xie, H., Zou, D., & Hwang, G. J. (2020). Application and theory gaps during the rise of Artificial Intelligence in Education. Computers and Education: Artificial Intelligence, 1, 100002.
11. Chung, E., Subramaniam, G., & Dass, L. C. (2020). Online Learning Readiness among University Students in Malaysia amidst COVID-19. Asian Journal of University Education, 16(2), 46-58.
12. Creswell, J. (2003). Research design: qualitative, quantitative and mixed methods approaches (2nd ed.). Thousand Oaks, CA: SAGE Publications.
13. Divekar, R. R., Drozdal, J., Chabot, S., Zhou, Y., Su, H., Chen, Y., … Braasch, J. (2021). Foreign language acquisition via artificial intelligence and extended reality: design and evaluation. Computer Assisted Language Learning, 35(9), 2332–2360. https://doi.org/10.1080/09588221.2021.1879162
14. Devitska, A. (2019). AR/VR/AI in Second Language Acquisition. https://dspace.uzhnu.edu.ua/jspui/bitstream/lib/55624/1/4-2_2019%20%281%29.pdf
15. Fan, Min, et al. (2020). Augmented Reality for Early Language Learning: A Systematic Review of Augmented Reality Application Design, Instructional Strategies, and Evaluation Outcomes.” Journal of Educational Computing Research, vol. 58, no. 6.
16. Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279-289.
17. Garzón, J., Pavón, J., &Baldiris, S. (2019). Systematic review and meta-analysis of augmented reality in educational settings. Virtual Reality, 23(4), 447-459.
18. Godwin-Jones, R. (2024). Distributed agency in language learning and teaching through generative AI. Language Learning & Technology, 28(2), 5–31. https://hdl.handle.net/10125/73570
19. Goh, C.S., Teoh, J. T., Chong, G. L., Tuan Sarifah Aini Syed Ahmad. (2023). “Mandarin AR” : An Augmented Reality Application for Mandarin Vocabulary Self-Directed Learning. Journal of Education and Practice,14(33),92-100. https://doi.org/7176/JEP/14-33-12
20. Hsu, T. C., Chang, S. C., & Liu, N. C. (2021). Applying artificial intelligence technology to support decision-making in teaching: a case study of non-native English student grouping. Interactive Learning Environments, 29(2), 164-176.
21. Huang, X., Zou, D., Cheng, G., Chen, X., & Xie, H. (2023). Trends, Research Issues and Applications of Artificial Intelligence in Language Education. Educational Technology & Society, 26(1), 112-131. https://doi.org/10.30191/ETS.202301_26(1).0009
22. Ibrahim, A., Huynh, B., Downey, J., Höllerer, T., Chun, D., & O’Donovan, J. (2018). ARbis Pictus: A Study of Vocabulary Learning with Augmented Reality. IEEE Transactions on Visualization and Computer Graphics, 24(11), 2867-2874.
23. Jamrus, M. H. M., & Razali, A. B. (2019). Augmented reality in teaching and learning English reading: realities, possibilities, and limitations. International Journal of Academic Research in Progressive Education and Development, 8(4), 724-737
24. Kazu, İ.Y., Kuvvetli, M. (2023). A triangulation method on the effectiveness of digital game-based language learning for vocabulary acquisition. Educ Inf Technol. https://doi.org/10.1007/s10639-023-11756-y
25. Kaiser H. F. (1974). An index of factorial simplicity. Psychometrika, 39, 31-36.
26. Karacan, C. G., &Akoglu, K. (2021). Educational augmented reality technology for language learning and teaching: A comprehensive review. Shanlax International Journal of Education, 9(2), 68-79.
27. Khalilova , L. ., Ravshanov , H., Xudayberdiyev, R. ., &To’rakulova , G. . (2024). Digital Technology Integration for Improving Foreign Language Learning. Eurasian Journal of Technology and Innovation, 2(1), 188–191. Retrieved from https://in-academy.uz/index.php/ejti/article/view/25672
28. Khoshnevisan, B., & Le, N. (2018). Augmented reality in language education: A systematic literature review. Adv. Glob. Educ. Res, 2, 57-71.
29. Kohnke L, Moorhouse BL and Zou D (2023) ChatGPT for language teaching and learning. RELC Journal 54(2): 537–550. https://doi.org/10.1177/00336882231162868.
30. Kondo, Tomotsugu. (2006). Educational Applications of Mixed Reality Technology and Current Issues. Technical Reports of the Institute of Image Information and Television Engineers, 30(58), 45-50.
31. Lee J.H., Shin, D. and Noh, W. (2023) Artificial intelligence-based content generator technology for young English-as-a-foreign-language learners’ reading enjoyment. RELC Journal 54(2): 508–516. https://doi.org/10.1177/00336882231165060.
32. Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901-918.
33. Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., &Daradoumis, A. (2022). Educational AI chatbots for content and language integrated learning. Applied Sciences, 12(7), 3239.
34. Md Ali, A., Mahmood, H., & Narawi, B. (2020). Current Trends of Augmented Reality in Malay.sian Education: A Systematic Review. International Journal of Advanced Science and Technology, 29(6), 2838-2851
35. Mohamad Z, (2024). Artificial Intelligence and the Future of Chinese Language Pedagogy: An In-Depth Analysis. In Zeynep Ç. K. &Abdulvahit Ç., Fostering Foreign Language Teaching and Learning Environment with Contemporary Technologies (pp. 1-20). USA IGI Global. https://doi.org/10.4018/979-8-3693-0353-5.ch001
36. Mertler,C.A.&Charles,C.M.(2014).Introductiontoeducationalresearch.London:PearsonEducation Limited.
37. Miyosawa, T., Akahane, M., Hara, K., & Shinohara, K. (2012). Applying augmented reality to e-learning for foreign language study and its evaluation. In Proceedings of the International Conference on e-Learning, e-Business, Enterprise Information Systems, and e-Government (EEE) (p. 1). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp).
38. Nghi, T. T., Phuc, T. H., & Thang, N. T. (2019). Applying AI chatbot for teaching a foreign language: An empirical research. International Journal of Scientific and Technology Research, 8(12), 897-902.
39. Parmaxi, A., & Demetriou, A. A. (2020). Augmented reality in language learning: A state‐of‐the‐art review of 2014–2019. Journal of Computer Assisted Learning, 36(6), 861-875.
40. Pokrivcakova, S. (2019). Preparing teachers for the application of AI-powered technologies in foreign language education. Journal of Language and Cultural Education, 7(3), 135-153.
41. Radianti, J., Majchrzak, T. A., Fromm, J., &Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147, 103778.
42. Santos, M.E.C., Chen, A.,Taketomi, T., Yamamoto, G., Miyazaki, J., Kato, H. (2014). Augmented Reality Learning Experiences: Survey of Prototype Design and Evaluation. IEEE Tans. Learn. Technology, 7, pp. 38-56.
43. Shadiev, R., Wu, T. T., & Huang, Y. M. (2020). Enhancing Learning Performance, Attention, and Meditation Using a Speech-to-Text Recognition Application: Evidence from Multiple Data Sources. Interactive Learning Environments, 28(6), 701-711.
44. Soomro, A. R., Tumrani, G. A., Bango, Z. A., &Maitlo, S. K. (2023). The Involvement of Artificial Intelligence (Ai) in Enhancing Communication Skills of English Language Learners. International Journal of Contemporary Issues in Social Sciences.ISSN (E) 2959-2461 (P) 2959-3808, 2(4), 937-944.
45. Sultan, A. Almelhes.(2023). A Review of Artificial Intelligence Adoption in Second-Language Learning [J].Theory and Practice in Language Studies 13,(5), 1259-1269.
46. Tan, C. K., Lee, K. W., & Ng, H. S. (2020). Augmented Reality to Enhance the Learning Experience in Cultural Heritage: A Case Study of the Bujang Valley, Malaysia. TEST Engineering & Management, 83, 12064-12071.
47. Tsai, P. H. (2019). Beyond second language acquisition: Learning as cognitive apprenticeship through technological assistance. Educational Technology Research and Development, 67(6), 1327-1351.
48. Woo, J. H., & Choi, H. (2021). Systematic review for AI-based language learning tools. arXiv preprint arXiv:2111.04455.
49. Yang, Q. F., Chang, S. C., Hwang, G. J., & Zou, D. (2021). Balancing cognitive complexity and gaming level: Effects of a cognitive complexity-based competition game on EFL students’ English vocabulary learning performance, anxiety and behaviors. Computers & Education, 164, 104114.
50. Yeo, M.A. (2023) Academic integrity in the age of artificial intelligence (AI) authoring apps. TESOL Journal. https://doi.org/10.1002/tesj.716.
51. Zuidema, P., Zuidema, W., & Grabe, W. (2021). The promise of educational technology: Reducing cognitive load and fostering engagement. Language Teaching, 54(3), 335-358.