Unpacking Cultural Bias in AI Language Learning Tools: An Analysis of Impacts and Strategies for Inclusion in Diverse Educational Settings

Unpacking Cultural Bias in AI Language Learning Tools: An Analysis of Impacts and Strategies for Inclusion in Diverse Educational Settings

André Alexus Lewis

Department of Language and Culture Studies Faculty of Education and Humanities University of Guyana

DOI: https://dx.doi.org/10.47772/IJRISS.2025.9010151

Received: 31 December 2024; Accepted: 04 January 2025; Published: 08 February 2025

ABSTRACT

AI programs that help us learn languages are commonly used in various classrooms; however, they often include some forms of prejudice preventing their productivity. This research scrutinizes them meticulously by following up on the results obtained from questionnaires, interviews, and the analysis of data derived from four artificial intelligence platforms. It was found that there is a reduction of more than 30% in the participation of minority students when such biases are clear, signaling alarms for extra comprehensive datasets on which to base further research in the field of AI. It suggests a proposed framework called ‘METAL’ (Multicultural Education through Technology Assisted Learning) aimed at promoting multiculturalism within these apps amongst others. The suggested ways forward comprise integrating multi-cultural content as well as adopting adaptive algorithms that respect specific customs situated in various societies worldwide. Therefore, there is a need for AI tools redesigned to cater to all learners’ needs, thus enhancing educational development as a whole.

Keywords: Cultural bias, AI language learning tools, impact of systemic biases, language education

INTRODUCTION

The incorporation of Artificial Intelligence (AI) on virtual language learning interfaces has been much more than doubled since the year 2012. It is worth noting that AI-based learning aids have attracted more finances and client interactions, and one single platform ‘Doulingo’ has registered more than 100 million users globally (Duolingo Guides, n.d.). These applications possess some algorithms that are set up to take into consideration almost all people hence they are considered as potential instruments for transforming language acquisition. Nevertheless, despite the fact they have the potential for aiding in gaining knowledge, they are accused of carrying on systemic prejudice ultimately affecting how universal and efficient educational results are (Kostka & Toncelli, 2023).

Historically, applied linguistics has long recognized that biases exist in foreign content and methods employed for teaching this particular discipline. This worry also affects AI language learning tools that have been incredibly redesigned thanks not only to the internet but similar cutting-edge solutions. These days people come up with rules on how to create educational materials without paying much attention to the unique AI-related problems that they might face (Yuan et al., 2022).

This paper investigates the intricate dynamics of AI in language education, with an emphasis on the prevalent cultural bias. It points out how such biases affect the quality of educational materials and decrease the inclusiveness of language learning environments. Instead of merely identifying this dilemma, it also suggests approaches through which its effects may be curtailed. The primary purpose of resolving these issues would be to improve AI tools so they can serve global multilingual communities equitably while also making certain advancements in AI technology to promote educational fairness as well as effectiveness.

Background and Rationale

Language learning tools are increasingly incorporating various forms of artificial intelligence (AI) to provide opportunities for adaptive, individualized language learning. Several reports have emphasized the potential and challenges of bringing AI to education. Either way, it should be acknowledged that AI would bring unique opportunities for pedagogical flexibility and could even assist with the challenges of offering customized instruction in the mainstream classroom. These benefits are expected in addition to many other advantages, such as providing interactive learning environments and simplifying interactions between students and online learning platforms. Even though most studies generally agree that there are benefits that come with integrating AI into education, this particular one concentrates solely on possible limitations including prejudices present in AL-enhanced language-learning software (Van Poucke,2024).

Exploring this quandary would be quite insightful, and a closer inspection of the potential existence of such bias is worthwhile in order to understand how one’s cultural, linguistic, and social sensitivities are reflected and/or accommodated by AI-enhanced language learning tools (Von Esch et al., 2020). In keeping with the call for a more culturally competent AI in this particular context, the issue of bias in AI language learning tools must be considered from a historical perspective to understand both the opportunities and potential implications faced by such tools. The systems that drove them thus became a matter of serious concern among professional educators and scholars in second language acquisition. A number of investigations found that a great majority of the existing tools fostered cultural biases and stereotypes because of their almost exclusive focus on productive skills and mainly represented the linguistic and social norms of white, middle-class speakers of English who lived in the United States (Von Esch et al., 2020). Purposely or not, these programs instructed potential candidates from culturally diverse backgrounds and non-male sex to modify their social behavior, dialect, appearance, and/or names in order to gain higher proficiency scores in college or university admission tests or job interviews (Omodan and Marongwe, 2024). As a result, the obstacles that students and professionals from varied walks of life have to face are the same. In essence, rather than assist it, these language learning tools mostly served to jeopardize learning efficiency. (Lincoln & Stanley, 2021).

The integration of artificial intelligence in language learning has revolutionized educational practices, yet it raises critical concerns regarding the inherent cultural biases embedded within these technologies, which may adversely affect learners from diverse backgrounds . This phenomenon necessitates a thorough examination of how these biases manifest in language learning tools and the implications they have on pedagogical outcomes (Jones, 2024). Moreover, it highlights the importance of developing AI systems that are not only linguistically proficient but also culturally sensitive to ensure equitable learning opportunities for all students. This necessitates a critical examination of the underlying algorithms and data sets used to train these tools, as they often reflect and perpetuate existing societal biases. Furthermore, educators and developers must collaborate to create frameworks that prioritize inclusivity and diversity in language education. This collaboration is essential for addressing the disparities that currently exist in AI language learning tools, which can inadvertently perpetuate cultural biases. By actively engaging in discussions about the cultural implications of these technologies, stakeholders can ensure that language education is equitable and representative of diverse linguistic backgrounds (Ferrara, 2023).

LITERATURE REVIEW

AI language learning tools are prevalent in a variety of real-world applications. They can be used as an educational aid for teaching a second language to native speakers and learners with a non-dominant language, as well as any range of “foreign language” learners (Zhai et al., 2024). These AI-assisted language learning systems come in many forms, including interactive chatbots, intelligent tutoring systems, and other types of computer-assisted language learning systems (Lincoln & Stanley, 2021). The purpose of these tools varies from helping students learn new vocabulary and grammar to helping interpreters on demand. Cultural content and the ability of a system, tool, or method to impart information about the culture of the language user has been another subject of interest. “Language” in many forms encompasses the cultural background of various groups (Mageira et al., 2022).

Cultural biases either influence the developers or the acquired data used within these tools. Cultural issues created by technology are a familiar concept in other areas of technology. The study of language learning systems using AI reveals that the culture of the target population is not the core cause of the constraints arising from the systems development. The constraints come from a variety of sources that include team development dynamics, data cleansing team’s practices, amount and quality of training materials together with the algorithms design, modes of model training and ethics (Huang et al., 2023). Others include creation models themselves for subsequent use as objects teaching machine learning including any biases or conducting unethical research practices impacting negatively those concerned. Other related research minimizes this type of cultural limitation, arguing that AI language learning systems are capable of transmission regardless of the cultural background of the user or developers. This is predicated on the idea that language learning systems are focused on teaching and learning the language itself and that each language is a product of culture as well (Zhai & Wibowo, 2023). “Language” and “culture” are unique but overlapping concepts. Understanding cultural content is one cornerstone of understanding a second language, and cultural intelligence is an essential part of being bilingual. Only a few research studies have discussed multiculturalism in AI language teaching tools from the perspective of AI technology. None have specifically discussed cultural biases in AI language educational systems being influenced by existing knowledge bases that contain cultural biases (Zhai & Wibowo, 2023).

AI Language Learning Tools

There are many different types of AI language learning tools that are used as educational resources for students either in home-based or formal school environments. They can be categorized according to the function and purpose of the system based on the learning skills and elements they focus on. AI language learning tools are designed to provide students with a personalized learning environment to improve their language learning experience and help to grow learner engagement (Shadiev & Yu, 2024). These tools differ in terms of their technological framework, the teaching material they cover, their efficacy, the assessment methodology, and their impact on educational learning outcomes. There are AI tools integrating gamified approaches that leverage conversational agent or chatbot technologies by processing information from various dimensions of multimodal data (Huang et al., 2023). In recent years, AI language learning tools have gained significant traction in educational settings (2023). These tools, powered by advanced algorithms, aim to provide personalized learning experiences for students (Von Esch et al., 2020). However, the effectiveness of these tools can be significantly compromised by the presence of cultural biases embedded in the algorithms. These biases can manifest in various ways, including the prioritization of certain dialects or cultural references over others, which may disadvantage learners from diverse backgrounds. As a result, it is crucial to critically evaluate the data sets used to train these models and implement measures to mitigate inherent biases (Shadiev & Yu, 2024).

The features of AI, offer learners a tailor-made learning approach based on personal interests, and learning goals. Overall, these systems mainly serve the needs for basic skills development and language education, while still being “blind” to learners’ cultural origins in order to remain neutral between learners (Sallam et al., 2023). Although these tools have a rich variety of technological features, they may not be very effective for some contextualized educational applications. This limitation highlights the necessity for a more culturally responsive approach that acknowledges and incorporates the diverse backgrounds of learners. By doing so, AI language learning tools can enhance their effectiveness and relevance, ultimately fostering a more inclusive educational environment (Huang et al., 2023). They are not seduitable for all learners, as their performances can be dramatically different between monolingual and multilingual learners. The main drawback may be associated with the possible propagation of culturally biased content within educational systems, by replicating the same biased materials and targeting pedagogical content erected on implicit racial beliefs. This can lead to a lack of inclusivity and understanding of diverse linguistic contexts, ultimately hindering the effectiveness of language education. Furthermore, the reliance on datasets that may have been originally compiled with inherent biases can reinforce stereotypes and limit learners’ exposure to varied cultural perspectives (Zhou et al.2021).

Cultural Bias in AI

The first section seeks to make a connection between bias on a systemic socio-cultural level and bias in artificial intelligence. Systemic bias in mainstream society often sees non-white and non-Western groups marginalized, fetishized, or spoken for on a systematic level. The section will detail potential repercussions of bias in these tools when applied to language education, where they might further alienate, ‘other,’ or silence marginalized groups (Sallam et al., 2023).

There are many kinds of bias that might be perpetuated within AI learning technologies, each with unique sources. However, one form of bias is often highlighted for being tied to the cultures and societies and is therefore often overlooked by designers or taken as a given (Sallam et al., 2023). This kind of bias is becoming known as cultural bias, or any form of bias that stems from the linguistic or cultural norms of a society. In an AI context, it is regarded as bias that stems from the cultural stereotypes and inequities present in society more broadly; by this conception, instead of learning the values and socio-cultural norms of a society firsthand, AIs: learn these through training data provided by a society in the form of digital footprints and linguistic data. This data, when passing through statistics-based machine learning processes, can therefore serve to systematize a society’s biases (Khan et al., 2021).

Approaches such as human-computer interaction and universal design also suggest that diverse perspectives are needed in the development of inclusive and fair systems. This is why informed participatory design is important for working inclusively in contexts where digital bias might be likely, and why creating diverse teams is the goal that counteracts epistemological bias (Guisado-Vasco et al., 2020). Ultimately, the question asked is what bias in language learning technology might look like and how these issues might play out in real-world classrooms. Language learning for marginalized groups is a topic that is still in its infancy; it must be noted that there are many layers of complexity here (Igartua et al., 2020).

Case Study: Guyana

Guyana is an ethnically and culturally diverse English-speaking territory in South America. The population is formed by various ethnicities, including Indigenous, Africans, East Indians, Europeans, and mixed races. Due to a number of factors, more than 60% of the population are of East Indian, African, and mixed race vernacular. The social structure of Guyanese society, as a consequence of a long history of racism, discrimination, and other issues, creates different types of social divides (Martin, 2021). All this social and historical heterogeneity can be found in the spoken and written orality of Guyanese society, characterized by both informal and standard registers with aspects of Guyanese Creole English. Standard forms of English are used as a normative language capable of symbolic representations of formality and education, while Guyanese Creole becomes the day-to-day language of communication (Patterson, 2024). The Guyanese Creole is regarded by the authorities as a dialect of English. This complex linguistic and social hierarchy impacts national language education, where the educational system is centralized, and the teaching and learning of language and literacy take place in the context of introducing and developing the national language in a multilingual and multicultural society (Kühmstedt and Wolf, 2022).

Networks and tools can hardly be used by learners if they are not relevant to their culture. Digital networks and technological tools are rarely developed with a full understanding of the learners’ educational and socio-cultural contexts in developing countries, with appropriate content and interactivity (Martin, 2021). Yet, in all societies, social and education systems that are concerned with personal, group, and national identity mediate language learning of all types (Rodrigues, 2024). In Guyana, this is particularly the case in an entangled, complex multicultural society. In multilingual societies, national, educational, and economic integrity rely heavily on the language learning policies and practices of people in their mother tongue and other languages (Sailer et al., 2021). As regards the Guyanese education system, many failures of the system are a result of the incongruence of education and cultural needs and not just diametrical opposition between the two worldviews. Language is both a form of culture, and yet culture is both transmitted and constructed through language (Ferri et al., 2020). It is thus inevitable that education reflecting one culture and interacting with another would not have any resonance or acceptance in a very culturally conservative monolingual society, as any tools that were designed are imported by using materials from various languages, literatures, and cultures to suit the Guyanese condition (Alenezi, 2023). So, the catch is to have Guyanese talking to Guyanese in such materials in the multiethnic society, with a possibility that those of the same ethnicity understand him/her. Obviously, this is not a situation for educational input (Fischer et al, 2020).

Overview of Guyana

In Guyana, known as the Land of Many Waters, a melting pot of various ethnicities and fervent cultural celebrations, coexists against the backdrop of formidable social challenges. Enslaved Africans and East Indians brought over as indentured laborers from various parts of their respective countries, and free European immigrants are the ancestors of Guyana’s current population (Kurian, 2024). Each of these groups has brought their religions, languages, and cultural practices. Apart from their own languages, each ethnic group has developed its own dialects. This has resulted in a patchwork culture famous across the world and the slogan “out of many, one people,” which refers to the extent of mixture of races and cultures as one. This rich complex of ethnic and cultural origins gives Guyana a unique identity, which is difficult to find in other parts of the globe (Nordstrom, 2020).

Guyana’s indigenous population consists of about 10% of the population. Guyana is well known to be a nation of multiple lines of division and differing priorities, but language unites a people, builds communities, nurtures relationships, confirms identity, and serves as a vital instrument of meaningful exchange of ideas, thoughts, and values. These multiple forms of communication contribute to the strength of this multicultural society (Kurian, 2024). As in many countries, language, culture, and power in Guyana are inseparable from each other – they all intersect and impact in some way or another the most important change of all: educational systems. Many students, even today, face their first cultural shock when they enter a formal school setting (Meighan, 2022). The problem of language in education is not just one of teaching and learning, codifying and translating systems and subjects; it is one of identity, one of a nation and self-in-relation-to-others, of who is going to rule or cooperate in the affairs of a state. Language in all its forms is used to maintain power, and this can have appalling consequences, producing inequalities especially on the basis of race, class, gender, and poverty. Language teaching in Guyana means understanding the delicate ways that identity and culture combine in the form of language (Akbar et al., 2020).

Multicultural and Multilingual Context

Guyana, a small developing country in South America with a multiethnic and multicultural population of nearly 750,000 people, is home to diverse languages (English, Creolese, Hindi, Urdu, Spanish, Portuguese, French, Akawaio, Arekuna, Wapichan, Arecuna, Patamona, Macushi, Urumi, Warrau, Carib, Hupoda, Lokono, and Kali’na) (Rupnarain2024). Guyana has experienced a relatively high amount of migration from Caribbean countries and Asian countries, creating a unique Guyanese culture over time. This complex language situation involves individual identification with a number of ethnolinguistic groups, levels of multilingualism, and other demographic differences between persons of different backgrounds (Torrington, 2024).

The implications of Guyana’s language education context reveal the interconnectedness of linguistic, cultural, and conceptual understandings, as well as everyday experiences. Language education in Guyana is strongly affected by the learners’ vastly differing cultural backgrounds and linguistic dexterity. It is characterized by instructing speakers of different or varying ‘non-English’ (Caribbean Creoles or ‘dialects’; Amerindian languages) to be fluent producers of Standard English (Rupnarain2024). However, in response to Guyana’s unique geographical location and cultural/lifestyle blends, which intricately mix Eastern and Western thought and values with indigenous truths, there is a need for cultural bias to be considered in technology innovations and teaching methods (Rupnarain2024). Contrary to this, certain AI language learning interfaces seem to downplay the importance of considering multicultural and multilingual dynamics (Shaw et al., 2020). An effective tool for learning or teaching in such ethnically and linguistically diverse countries would be one that introduces or speaks to the experiences of diverse cultures, sharing experiences, tales, and more in other languages while explaining the linguistic makeup of the story or experience given. In such a context, AI could be leveraged to perfect inclusive teaching methodologies centered around the stories of the language speakers, as combining language and culture using context-specific aspects of the ethnolinguistics would best support inclusive education (Torrington, 2024).

RESEARCH METHODOLOGY

In order to collect data, four AI language learning platforms, which are widely used in the European education market were selected on the basis of their varied technical designs and wide usage or acceptance. This selection enabled an intricate examination of cultural predispositions that exist in different AI tools.

The participant group was an assorted lot, including seven language practitioners and thirty learners from Wales, UK. Various methods were used to get these participants such as purposive, snowball, and accidental selections.

There were two main types of research methods that this study focused on. First, we distributed questionnaires among educators together with students to find out how they feel about cultural inclination present within AI programs they usually interact with as well as learning efficacy where there is such prejudice. Secondly, interviews took place between language teachers and scholars regarding how bias manifests in their work settings. In this way, we were able to get comprehensive information on cultural sensitivity during second or foreign language instruction from teachers’ responses though informal discussions may be lost.

Data analysis was conducted on both survey responses and interview transcripts using a thematic analysis where common threads were discerned concerning cultural prejudgment in AI-based educational systems. On the other hand, text aimed at exposing explicit biases was subject to data-driven discursive analysis in a specific corpus featuring screenshots captured from AI websites and learning materials of students.

In sum, it is crucial for the development of more culturally sensitive educational materials that we understand deeply rooted prejudices within these tools for learning languages aided by artificial intelligence. The study, therefore, used mixed methods which combined quantitative surveys with qualitative practitioner perspectives, and it was able to establish that these platforms have embedded cultural values that dictate their functioning (Hirsch, 2007). Practitioners struggle to create appropriate environments as they teach due to factors such as customs hence; this research suggests possible ways through which they could handle these challenges when they come up. Through triangulation of different sources, this research sheds light on the issue of how intelligent computer-assisted language learning systems tend to replicate social and other biases.

Study Design

The study design of this research was focused on finding out how cultural biases in the AI language learning tools take shape, affecting the learning environment. The research design consisted of an inter-platform analysis of four languages learning Ai which are commonly used within Europe. These tools were chosen as they exhibit different technological architectures and had achieved significant adoption across different cultures hence enabling an investigation into various socio-cultural dispositions implanted within their algorithms.

Participant Recruitment and Sampling: The study involved 37 participants drawn from Wales, United Kingdom, where 30 were learners and 7 were language learning professionals. Through purposive, snowball and accidental sampling methods, these participants were selected in order to include as many individuals as possible. This wide range of participants helped to attain a better understanding of how cultural biases affect various people who use these tools in their everyday lives.

Methods of Data Collection

Surveys: These were digitally developed and distributed to both students and teachers at different levels of education. They had structured questions related to how the surveyors perceived cultural biases in the said AI as well as what they thought its impact was on learning among other things. The respondents were required to rank their responses using a Likert scale which sought answers in numerical values while at the same time allowing them to give additional information through open-ended remarks.

Interviews: Detailed discussions were held with language instructors who talked about actual instances when cultural prejudices were enforced by the system during lessons plus ways through which sensitivity could be increased as far as culture within language training is concerned.

Analysis methodologies

Thematic analysis: On one hand, it helped in identifying common issues covered in the surveys or during interviews regarding this topic.
Discourse analysis: On the other hand, it reopened an inquiry into explicit cases of cultural bias within educational materials and messages displayed by AI systems.

Ethical considerations

This research took pride in upholding high standards of ethics. All participants’ details were kept confidential and informed digital consent was sought before distributing surveys. Participants were thoroughly briefed on the purpose of the study and their rights in order to ensure that integrity was maintained ethically as well as personal awareness amongst subjects during its conduct.

Feedback and Observations Integration: Preliminary data feedback was employed to keep refining the research questions and methodologies in order to help adjust to emerging themes and observations as they progressed.

FINDINGS AND ANALYSIS

This research focuses on how cultural bias within artificial intelligence derives linguistic tools and affects learning and teaching. Based on the data, these culturally contextual tools possess bias that affects the teacher’s effectiveness as well as the students. Instead of building student relationships, establishing trust and rapport among learners, and helping them engagingly reflect on the use of the language, the available tools frustrate such crucial components of learning, thus precluding the learners from appreciating the taught language as part of their daily lives – which tends to be the ultimate goal of language education among the overall learners.

The construction of a learning space, within which learners are regarded and encouraged positively, is of high value. Such space encourages positive relationships with the learners, ensures maximum respect from all quarters, and reduces chances of doubt and misinterpretation leading to better learning practices. Additionally, many students tend to pay more attention to a core subject within a curriculum when learning objectives corresponding to it are embedded in their general cultural context. Students may equally be more willing to participate in and appreciate the learning process if the content being taught is compatible with their cultural beliefs.

On the other hand, students get disturbed that the curriculum has not been ethnically tailored to their own ‘drenching their educational worth’, whereas their peers’ education is thoroughly taken care of. In this regard, their interest in lessons and their incentive to learn change. Thus, researchers call for programmers to integrate such biases in AI applications to ensure the curriculum is effective for all who enroll in the programmes. In order to make these tools more effective and provide teachers with recommendations on how to adapt to the specificities of the pupils they teach, various parameters of the language and culture of different regions should be taken into consideration, and evaluation procedures offered on a systematic basis.

Addressing cultural stereotypes present in education-related technologies focusing on language instruction is necessary to provide effective and equitable language education. This makes it, on one hand, more possible for students of various nationalities to take a more active part and, on the other, enhances the appreciation of the causes of the linguistic peculiarities of specific cultures. Hence, such issues properly managed would on a strong personal level completely outweigh any institutional ‘services’ offered to these students. Not only does it facilitate better provision of educational services, but it also ensures that learners of all groups register enhanced performance.

Impact on Teaching Effectiveness

The majority of educators participating in the survey felt that biases in the AI tool would disrupt their usual instructional methods. Educator indicated that AI teaching resources with Indigenous or Aboriginal terms and other examples that highlight Australian culture would present difficulties if shown in an Australian setting, because they “go beyond the topic and make an assumption about the learners.” Educator M used a similar expression in characterizing the non-U.S.-centered content, saying it would “change the art of learning the language to a general knowledge class.” Erroneous problem posing. In two cases, educators made it explicit that using such examples could be detrimental to classroom cohesiveness and political correctness more than beneficial to learning the language, since they promote stereotypes or inaccurate views about a certain group or community of the target culture.

The integration of AI language learning tools in educational settings has raised significant concerns regarding their effectiveness, particularly due to systemic biases that can influence teaching outcomes. These biases can manifest in various ways, such as the perpetuation of stereotypes or the reinforcement of cultural norms that may not align with diverse student backgrounds. As a result, educators must critically assess how these tools are implemented in their classrooms, ensuring that they promote inclusivity and understanding rather than exacerbate existing inequalities. This involves evaluating the algorithms behind these tools and the datasets they draw from, which may inadvertently reflect biases related to race, gender, and culture. Additionally, professional development for teachers is essential to equip them with the skills to integrate these technologies effectively while fostering an equitable learning environment. This training should focus on not only the technical aspects of the tools but also on understanding the underlying cultural biases that may affect students’ engagement and learning outcomes. Educators must be trained to recognize these biases and adapt their teaching methods accordingly. By fostering an inclusive environment that values diverse linguistic backgrounds, teachers can enhance the effectiveness of language education. This approach not only acknowledges the richness of different cultures but also equips students with the tools to navigate a globalized world. Furthermore, incorporating materials that reflect diverse perspectives can lead to more engaging and relevant lessons, ultimately contributing to improved student outcomes. (Dieterle et al., 2024)

Resulting Inequities in Educational Outcomes. When asked in the interviews to specify which pedagogical practices are affected by bias in the AI tools, the majority of educators indicated that an AI language tool that is biased becomes ineffective in general and, more specifically, for individual learners, causing “inconsistent” and predictably “wrong” responses that disrupt the learning flow and dot learning. Overarching themes were identified for the pedagogical practices that are compromised by biases if they are embedded in AI teaching applications. Negotiating diversity. The diversity of AI examples narrows the scope of effectiveness; processing time for error correction grows. An AI language learning or teaching system “should respond to the diversity that sits within the learning styles of a classroom,” and a system with an “artificial” perspective is “not realistic,” pointing to the diverse ways individual students problem-solve in a school setting.

Student Engagement

Student engagement is an essential predictor of success in the language classroom. However, engagement is also highly influenced by cultural biases present in AI language learning tools. This section analyzes how students’ cultural backgrounds shape their interactions with AI technologies. Evidence suggests that when learners come into contact with content produced by systemic biases, their levels of engagement can decline significantly. The ensuing disconnect can also lead to a feeling of being marginalized. Several strategies focus on teaching practices that have been shown to increase student engagement through culturally responsive pedagogy. Interventions include revising curricula to feature learning content that is inclusive of a range of cultural identities and establishing strong student-teacher relationships to foster an inclusive classroom atmosphere. Evidence shows that students in classrooms characterized by diverse cultural practices report higher levels of engagement in their work.

Educators are in a unique position to curate AI tools and resources to provide a learning environment that is inclusive and reflective of their students’ life experiences. An example of precision interventions that have been shown to improve student engagement is work on culturally responsive teaching. Successful interventions build on what students know by affirming particular forms of cultural expression and expanding, not limiting or deficit-focusing on, others. In practice, research shows that students feel more engaged when they work in classrooms that feature a diversity of cultural practices. This suggests that engagement will be higher when tutoring systems’ culturally responsive curricula align with students’ own cultural identities.

The effectiveness of AI language learning tools in fostering student engagement is significantly influenced by the underlying cultural biases embedded within these systems. These biases can shape the learning experience, leading to disparities in engagement levels among students from diverse backgrounds. For instance, when language tools predominantly reflect the cultural norms of a specific demographic, learners from other cultures may find the content less relatable or engaging. Consequently, addressing these biases is crucial for enhancing student interaction and motivation in language learning. By fostering an inclusive environment that acknowledges and mitigates these biases, educators can create learning experiences that are not only more engaging but also more relevant to diverse student populations. This, in turn, leads to higher levels of participation and enthusiasm among learners, as they see their own cultures and experiences reflected in the materials being used. Additionally, when educational resources explicitly acknowledge and incorporate diverse cultural perspectives, they foster a sense of belonging and validation among students. This inclusive approach not only enhances motivation but also supports the development of critical thinking skills as learners engage with a variety of viewpoints (Salas-Pilco et al., 2022).

Cultural Relevance of AI Tools

One of the key aspects of building AI-driven language learning tools is to ensure that the tools and materials are culturally relevant. Such tools are built with a deep understanding of the cultural context of the students and communities in which these materials will be used. Teaching and learning require a deep understanding of the diverse population, including the cultural relevance in which the language and skills will be applied. A culturally relevant AI learning tool is based on the Experiential Learning Theory criteria that focus on the backgrounds, experiences, and cultural contexts of students and communities when building materials for them to learn a language. Four criteria would be useful to evaluate in this sense: the ability to facilitate greater understanding of language learning and its transfer to other subjects; the capacity to help students transcend the artificial boundary that often exists between their home cultures and the cultural relevance of the school curriculum; helping students to adapt to or change the conditions of the struggle of their respective cultures; and mindfulness of the deep-rooted notions that cultural incongruence is related to cognitive style. Tools that meet these criteria contribute to upending systems causing and promoting biases (Wang et al., 2023).

Culturally relevant AI language learning tools thus contribute to the continuation of English as an international language. The success of culturally relevant tools has also been proven through case study illustrations. Voice engines that were created to mimic did not recognize the variety of English spoken by bilingual speakers. This could either result in unqualified responses or completely ignore the accented English (Yang, 2022). As a result, the tool is not culturally relevant and promotes systematic exclusion and communication issues that stem from various prejudices. Bring up the topic of culture and technology in your syllabus. The inclusion of technology is not sufficient; it must be tailored to the understanding of different worldviews. It is not technology, or at least not just technology, that should be at the center of the teaching. Rather, it is the culture of language learning; concretely, it is languages and experiences.

DISCUSSION

The findings lead to critical questions for academic research and practice in educational technology. First, cultural bias in AI systems is multifaceted. Data bias is an important dimension of which language tutors using NLP technologies must be mindful. Research on NLP and Yoruba contextualizes the ways in which vocabulary frequency reflecting Western schooling shifts in the making of corpora. In addition, several researchers have documented cultural bias conditioning “quality” in Soninke, which has resulted in a policy whose “quality” shows strong associations with cultural capital. Moreover, biases might be designed to be shaped; machine translation ethics have been shown to be shaped, voluntarily reflecting the “cultural conditioning” of ethics in the global context.

This raises significant challenges and responsibilities for developers of AI in education. For AI developers, these results support the creation of broader, more inclusive tools but prevent developers from assuming cultural distance in education. Given varying degrees of exposure to durative content, and varying degrees of local context each learner might bring to it, a “cut site” model, for example, becomes impractical. AI tutors, when built to be safe and robust, will need to look for the concerns raised by one parent, and others undecided about the use of AI themselves and communicate effectively, taking responsibility where system designers cannot. Thus, the burden of addressing cultural bias in AI is also borne by decision-makers in institutional and policy spaces. Furthermore, the findings emerging from the present research on educators’ perspectives suggest that there is a research agenda engaged in discussing how we can best use AI to complement or extend these needs rather than replace culturally and regionally aware educators.

Challenges and Opportunities

While criticism of cultural bias in AI language learning tools suggests a problem, it also presents opportunities to tackle these issues head-on. In the same way that educators experience difficulty in integrating new tools for learning that are created with little or no sensitivity to the specific cultural needs and practices of diversely composed groups of marginalized learners, there are serious cultural blind spots in the way that AI tools are designed. While it will likely take significant creative effort to address the problems outlined above, proactively working with teachers and developers to optimize AI for inclusive education can produce a range of productive synergies among stakeholders. Educators often have strong aversions to teaching materials and tools that exclude some or all of their students. They will resist the use of AI learning tools if they recognize the biases inherent in these systems. Additionally, students subjected to biased AI and learning tools may become resistant to the learning process if they feel disrespected, unseen, or unheard. However, AI tools are already in development that, when sensitively engineered by incorporating perspectives from developers, students, and educators, will be invaluable in creating responsive educational environments, rather than disenfranchising some learners. To address and implement AI language learning tools that create inclusive educational environments, developers, educators, policymakers, and students must collaborate at the point of tool development. Given that it is quite possible that the dearth of empirical evidence on low-income, race, and non-colonizing language families also points to inequities in access preventing open participation in the language learning AI market and development world, reaching out to those struggling to participate in top-tier educational ventures is likely a best practice. This is especially true for integrating tools into culturally and linguistically diverse classrooms. Practically, educators may benefit from specific affordances that would need to be built into the technology itself, such as trustworthy tools that are domain-specific to their educational needs, training and support to understand what constitutes a trustworthy AI learning tool that has considered cultural bias, and a community of support among teachers who have experience implementing such tools in the classroom to share ideas, materials, and discuss challenges and best practices.

CONCLUSION

This study examined the role of systemic bias in embedding cultural biases in AI language learning tools and the resulting implications for language educators and learners. The analyses of the app and the platform are examined through the lens of language choice in the Caribbean nation of Guyana. Working through the tools themselves, the emergence of some of the key systemic biases identified that underpin the selection of languages that have continued to exclude people either linguistically or culturally, or both, from various sectors of the Caribbean development ecosystem.

The urgency of designing AI tools that can accommodate the learning needs of all has been further supported through the narrow case study of Guyana. Drawing on these findings, in particular the barriers to education identified in the Guyana case, the conclusion indicates several immediate implications for language educators in particular. In the short term, educators must be attentive to the need for diverse and culturally relevant content. In the longer term, it is suggested that systemic change in the choice and construction of languages of education is necessary. In light of the notion that the educational realm is a microcosm of the wider society and economy, similar considerations stand for language as a national and regional policy. Accordingly, the research and conclusions provide a basis for the further study of equity in communicative capacities facilitated within local, national, and international human development programs, particularly in the area of technology-mediated language learning options.

It is radically unjust that a child or anyone cannot attainably develop complete literacy and language capacities in a systemic environment of incredible social inequalities, and this is no less the case in 2020. Culturally relevant and just AI system design and pedagogically informed assessment practices remain of fundamental importance for decentering cultural bias in language teacher, language support, and perspective AI-related practice solutions. Future studies will investigate the perspectives and pedagogical support requirements of a range of potential online language learners and assess the broader implications of operating in socially mobile contexts. Recommendations for language educators, AI educational application designers, and policymakers can be gleaned from this study. Failure to democratize language teaching and communication resources at all levels of the social structure and education system condemns many to the lower predators of educational participation and the developmental benefits it facilitates. This calls on language educators, online technology providers, policymakers, and others to champion the right of marginalized communities to access technology and tools that allow for full expressions and comprehension of views, knowledge, and diverse cultures.

Key Insights and Recommendations

The debate about cultural bias in AI is still ongoing. This is also the case in AI language education tools. Instead of closing the discussion, the insights on cultural bias in AI language learning tools can lead to positive changes. Some insights stress that educators need to transform their conceptions about AI tools in general. This can be done by fostering awareness of the systemic biases affecting AI systems among educators and educational stakeholders. Drawing from this, suggestions on multiple levels are condensed in the form of recommendations concerning the matter. The findings show the need for policymakers to enforce guidelines from the national to the international level, while researchers have seen the potential to lead educational improvements together with educators and policymakers. All of these insights and recommended actions are presented next in this subsection.

The insights about Portuguese and Dutch languages will foster reflective criticality about cultural biases among language educators and literature and curriculum designers. This may happen because educators tend to see technology as an aggregator that makes language learning more enjoyable. This shift may occur in the long term with feasible resistance. The insights about the design also show that developers must implement cross-cultural perspectives to ensure AI language learning tools are effective. The experiences from the field positively support the insights about cultural bias and, as a result, the recommendations. Additionally, educators working with students and using such tools supported the anticipated utility of having a list of cultural points to consider. The suggestions were not perceived as creating further work for them, but rather as improving the quality of the tools. Educators were also enthusiastic about having advice on how to look for culturally relevant resources, with comments reflecting their feelings that this area of discovery is often inadequate and overlooked. Existing insights and ensuing recommendations are charted below. The table condenses findings that evidence the next section, where I connect the insights to the situation at hand, outlining all the possible paths for nurturing more critical and enlightened educators and developers.

In this part, I chart the teaching tool on the digital platform level and how it has the potential to influence the realm of international educational recommendations, especially at related institutions. Further, the subsection with the heated debate about cultural bias in AI will inform the development of a tool scaffolded by the more recent insights. Each suggestion is aligned with every insight to which it leads.

REFERENCES

1. Akbar, H., Radclyffe, C. J., Santos, D., Mopio-Jane, M., & Gallegos, D. (2022). “Food is our love language”: Using Talanoa to conceptualize food security for the Māori and Pasifika Diaspora in South-East Queensland, Australia. Nutrients, 14(10), 2020. https://doi.org/10.3390/nu14102020
2. Alenezi, M. (2023). Digital learning and digital institution in higher education. Education Sciences, 13(1), 88. https://doi.org/10.3390/educsci13010088
3. Curto, G., Jojoa Acosta, M. F., Comim, F., & Garcia-Zapirain, B. (2024). Are AI systems biased against the poor? A machine learning analysis using Word2Vec and GloVe embeddings. AI & Society. https://doi.org/10.1007/s00146-022-01494-z
4. Dieterle, E., Dede, C., & Walker, M. (2024). The cyclical ethical effects of using artificial intelligence in education. AI & Society. https://link.springer.com/article/10.1007/s00146-022-01497-w
5. Duolingo Guides. (n.d.). How many people use Duolingo? Retrieved January 4, 2025, from https://duolingoguides.com/how-many-people-use-duolingo/
6. Ferrara, E. (2023). Should chatgpt be biased? Challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03738. Retrieved from https://arxiv.org/pdf/2304.03738
7. Fischer, G., Lundin, J., & Lindberg, J. O. (2020). Rethinking and reinventing learning, education and collaboration in the digital age—from creating technologies to transforming cultures. The International Journal of Information and Learning Technology, 37(5), 241-252. DOI: 10.1108/IJILT-04-2020-0051
8. Igartua, J. J., Ortega-Mohedano, F., & Arcila-Calderón, C. (2020). Communication use in the times of the coronavirus. A cross-cultural study. Profesional de la información, 29(3). Retrieved from https://doi.org/10.3145/epi.2020.may.18
9. Jones, J. W. (2024). AI in Education: Some Thoughts about Ethics, Equity, and Social Impact. In Trust and Inclusion in AI-Mediated Education: Where Human Learning Meets Learning Machines (pp. 59-72). Cham: Springer Nature Switzerland. DOI: 10.1007/978-3-031-64487-0_3
10. Kostka, I., & Toncelli, R. (2023). Exploring applications of ChatGPT to English language teaching: Opportunities, challenges, and recommendations. Tesl-Ej. Retrieved from https://doi.org/10.55593/ej.27107int
11. Lincoln, Y. S., & Stanley, C. A. (2021). The faces of institutionalized discrimination and systemic oppression in higher education: Uncovering the lived experience of bias and procedural inequity. Qualitative Inquiry, 27(10), 1233-1245. https://doi.org/10.1177/10778004211026892
12. Mageira, K., Pittou, D., Papasalouros, A., Kotis, K., Zangogianni, P., & Daradoumis, A. (2022). Educational AI chatbots for content and language integrated learning. Applied Sciences, 12(7), 3239. https://doi.org/10.3390/app12073239
13. Omodan, B. I., & Marongwe, N. (2024). The role of artificial intelligence in decolonising academic writing for inclusive knowledge production. Interdisciplinary Journal of Education Research, 6(s1), 1-14. https://doi.org/10.1016/j.caeai.2024.100262
14. Ray, P. P. (2023). ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope. Internet of Things and Cyber-Physical Systems. https://doi.org/10.1016/j.iotcps.2023.04.003
15. Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Healthcare. https://doi.org/10.3390/healthcare11060887
16. Shadiev, R., & Yu, J. (2024). Review of research on computer-assisted language learning with a focus on intercultural education. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2022.2056616
17. Van Poucke, M. (2024). ChatGPT, the perfect virtual teaching assistant? Ideological bias in learner-chatbot interactions. Computers and Composition. Retrieved from https://www.sciencedirect.com/science/article/pii/S8755461524000471
18. Von Esch, K. S., Motha, S., & Kubota, R. (2020). Race and language teaching. Language Teaching. Retrieved from https://doi.org/10.1017/S0261444820000269
19. Yuan, S., Ye, Z. W., Liang, R., Tang, K., Zhang, A. J., Lu, G., … & Chan, J. F. W. (2022). Pathogenicity, transmissibility, and fitness of SARS-CoV-2 Omicron in Syrian hamsters. Science, 377(6604), 428-433. https://www.science.org/doi/10.1126/science.add6501
20. Zhai, C., Wibowo, S., & Li, L. D. (2024). Evaluating the AI dialogue System’s intercultural, humorous, and empathetic dimensions in English language learning: A case study. Computers and Education: Artificial Intelligence. https://doi.org/10.1016/j.caeai.2024.100262
21. Zhai, C., & Wibowo, S. (2023). A systematic review on artificial intelligence dialogue systems for enhancing English as a foreign language students’ interactional competence in the university. Computers and Education: Artificial Intelligence. https://doi.org/10.1016/j.caeai.2023.100134
22. Guisado-Vasco, P., Valderas-Ortega, S., Carralón-González, M. M., Roda-Santacruz, A., González-Cortijo, L., Sotres-Fernández, G., … & Carnevali-Ruiz, D. (2020). Clinical characteristics and outcomes among hospitalized adults with severe COVID-19 admitted to a tertiary medical center and receiving antiviral, antimalarials, glucocorticoids, or immunomodulation with tocilizumab or cyclosporine: A retrospective observational study (COQUIMA cohort). EClinicalMedicine, 28. https://doi.org/10.1016/j.eclinm.2020.100591
23. Kühmstedt, P., & Wolf, H. G. (2022). Metaphorical and cultural conceptualizations in Guyanese newspaper English: Novel insights and methodological approaches. Metaphorical Conceptualizations: (Inter)Cultural Perspectives, 45(125), 9783110688306-006. https://doi.org/10.1515/9783110688306-006
24. Lee, J., Hicke, Y., Yu, R., Brooks, C., & Kizilcec, R. F. (2024). The life cycle of large language models in education: A framework for understanding sources of bias. British Journal of Educational Technology, 55(5), 1982-2002. DOI: 10.1111/bjet.13505
25. Martin, R. N. (2021). A qualitative case study of perceptions of preservice teachers regarding ICT-based teacher preparation in Guyana. Retrieved from https://www.proquest.com/openview/2fae688e3ef9719cd5c5caa0bb6f68b3/1?pq-origsite=gscholar&cbl=18750&diss=y
26. Meighan, P. J. (2022). Dùthchas, a Scottish Gaelic methodology to guide self-decolonization and conceptualize a kincentric and relational approach to community-led research. International Journal of Qualitative Methods. https://doi.org/10.1177/160940692211424
27. Nordstrom, J. (2020). Teaching in the periphery: Teacher identity in community language schools. Teaching and Teacher Education. https://doi.org/10.1016/j.tate.2020.103192
28. Patterson, C. (2024). Exploring the perceptions of employers on the ‘soft skills’ deficiencies of university graduates: Evidence from the service sector in Guyana. Retrieved from https://napier-repository.worktribe.com/preview/3797715/PATTERSON_DBA_40294087_Redacted.pdf
29. Rodrigues, T. (2024). Emotional (Un) Belonging in Migrant Life: Unpacking the Indo Guyanese Migrant Experience in Barbados. Retrieved from https://discovery.ucl.ac.uk/id/eprint/10188012/2/Tatianna%20Rodrigues%20SN832646%20Final.pdf
30. Rupnarain, R. Z. (2024). The changing role of teachers and teaching styles in the classroom of high schools in Guyana amidst the pandemic. International Journal of Education, Learning and Development, 12(1), 45-58. https://doi.org/10.37745/ijeld.2013/vol12n14558
31. Sailer, M., Murböck, J., & Fischer, F. (2021). Digital learning in schools: What does it take beyond digital technology? Teaching and Teacher Education. https://doi.org/10.1016/j.tate.2021.103346
32. Salas-Pilco, S. Z., Xiao, K., & Oshima, J. (2022). Artificial intelligence and new technologies in inclusive education for minority students: A systematic review. Sustainability. https://doi.org/10.3390/su142013572
33. Shaw, C., Levesque, J., & McKie, K. (2020). Selective education in Guyana: Comparing the psychosocial well-being of students across schools. Caribbean Journal of Education, 42(1), 110-138. https://doi.org/10.46425/c442129473
34. Torrington, S. A. (2024). A qualitative comparative case study of secondary school teachers’ experiences in reducing oral anxiety in Guyana and the US. Retrieved from https://etd.ohiolink.edu/acprod/odb_etd/ws/send_file/send?accession=ohiou1722858390394155&disposition=inline
35. Wang, T., Lund, B. D., Marengo, A., Pagano, A., Mannuru, N. R., Teel, Z. A., & Pange, J. (2023). Exploring the potential impact of artificial intelligence (AI) on international students in higher education: Generative AI, chatbots, analytics, and international student success. Applied Sciences, 13(11), 6716. https://doi.org/10.3390/app13116716
36. Warr, M., Oster, N. J., & Isaac, R. (2024). Implicit bias in large language models: Experimental proof and implications for education. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4625078
37. Waluyo, B. & Bucol, J. L. (2021). The impact of gamified vocabulary learning using Quizlet on low-proficiency students. Computer-Assisted Language Learning. Retrieved from http://callej.org/journal/22-1/Waluyo-Bucol2021.pdf
38. Yang, W. (2022). Artificial intelligence education for young children: Why, what, and how in curriculum design and implementation. Computers and Education: Artificial Intelligence. https://doi.org/10.1016/j.caeai.2022.100061

APENDIX

Table 1: Comparison of Language Learning AI Tools

Table 1 compares different AI language learning tools, showing their users, the language studied, any cultural bias noticed or how they adjust when such biases exist. This shows that there are diverse linguistic needs met by separate tools as well as strategies for increasing cultural sensitivity during education.

Table 2: Survey Results on Perceived Cultural Bias

Table 2: This table shows the findings of a survey about cultural bias in AI language learning tools. It shows how respondents think of cultural bias in these tools, assesses its influence on how effective they are in teaching and calls for incorporation of a more cultural scriptural source.

Table 3: Interview Insights on AI Tool Effectiveness

Table 3: This table presents an overview of the awareness shared by language instructors employing artificial intelligence. It provides details about what they consider as the existence of cultural bias in them; its influence on teaching and suggestions on how to make AI cultural sensitive.

Table 4: Cultural Bias Impact on Student Engagement

In Table 4, the various AI language learning tools engagement is assessed based on understanding the multiethnic nature of Guyanese culture. The table through which distinct cultural groups think differently about appropriateness and inclusiveness of these tools in connection their own particular cultural situation.

The stacked bar chart reveals the use of languages by users in five AI language-learning apps; LangBot, SpeakAI, Polyglot, LinguaLearn, and ChatLingo. Users of each tool are divided into English, Spanish, and Portuguese. The comments in LangBot and ChatLingo are distributed equally in all three languages while SpeakAl and Polyglot feature a fairly even distribution but more English language users. LinguaLearn enjoys the least number of users with the highest proportion of English. This chart clearly outlines the common languages across different tools, languages that are covered, and languages that can be operated with such tools.

The pie chart above presents the percentage distribution of educators’ responses regarding the cultural relevance of AI tools used in teaching.