Using Gamification Technique in the Assessment of Executive Function of Kindergarten Pupils

Using Gamification Technique in the Assessment of Executive Function of Kindergarten Pupils

*Karen Gan Pineda, Dondon B. Buensuceso, Ph.D.
FEU Roosevelt, Cainta, Rizal, Philippines
*Corresponding Author

DOI: https://dx.doi.org/10.47772/IJRISS.2023.71078

Received: 10 September 2023; Revised: 22 September 2023; Accepted: 25 September 2023; Published: 23 October 2023

ABSTRACT

Early assessment of Executive Function (EF) can have great implications to children’s academic success. However, there is a need for assessment tools that are adapted to the skills of today’s preschool children other than paper pencil tests. This paper looked at the possibility of using gamification by using the mobile phone app Digital Trail Making Test by Simone Sacchi (dTMT-SS) in the assessment of the Executive functioning of kindergarten pupils at the height of the pandemic. This quasi experimental research utilized the pre-test and post test on ten (N=10) kindergarteners, with a mean age of 6.28 who remotely performed the dTMT at their homes with their parents who were instructed on how to administer or play this app. The time stamped results indicated an average level for both Part A (Numbers only) and Part B (Numbers and letters) in the pretest shows a mean of 1 minute and 10 seconds and 2 minutes and 37 seconds, respectively. The post test mean for Part A mean was 1 minute and Part B mean was 2 minutes and 16 seconds which both indicate average level. When comparing the results there is significant difference in the level of EF for both Part A (Numbers only) and Part B (Numbers and Letters). It can be inferred that the use of gamification technique is an effective tool in the assessment of EF in kindergarten pupils of today’s generation. The use of gamification in assessment is a faster, easier tool especially during health scares such as the Covid19 pandemic. It is recommended that more testing should be done before classes begin as part of the assessment for admission to kindergarten on a larger population and in both public and private schools.

INTRODUCTION

Some children get frustrated and throw fits when they have difficulty in beginning a task. Others would start and not finish their work at all. Some have a hard time remembering and following multi-step instructions. All these are associated with Executive Function (EF) and are often related to Autism, Attention Deficit Hyperactivity Disorder (ADHD) and even depression. EF skills are higher-level cognitive skills that children need in performing daily tasks and in academics. EF skills should be taught and practiced early so that children can cope with everyday tasks and have a better chance of academic success. How would you know that your child lacks EF skills? Assessment is key to determining whether your child has difficulty in EF. Early assessment can have great implications for children’s academic success.

However, there is a need for assessment tools that are adapted to the skills of preschool children (Nieto et. al., 2016). Assessment tools that can assess the EF skills of today’s children who are digital natives.

The aim of this quasi-experimental study was to examine the use of the gamification technique in the assessment of EF skills of kindergarteners prior to any diagnosis in order to enhance or strengthen their EF skills rather than use it for treatment or intervention. This approach entailed the use of a game application (app) that measures the executive function tasks of preschool children (Carlson & Zelazo, 2014), specifically the Digital Trail-Making Test (TMT). This app is the touchscreen device for the neuropsychological test called “Trail Making Test” which introduces new experimental features.

Background of the Study

Back in 2008, this researcher came to know about executive function and that this part of the brain is responsible for how to get things done. It was associated with children with Attention  Deficit  Hyperactivity Disorder (ADHD) and Autism. Difficulty in getting things done is one of the symptoms of ADHD and Autism.  Back then, myths spread that children had these conditions because of childhood vaccinations for measles and others. Knowing about EF, early assessment, and strategies can help children cope with their difficulties and be successful in their academic life.

Here in the Philippines, most children get assessed by teachers when they enter Kindergarten, which is the mandatory entry stage to primary education.  Section 2 of this Act states that all five (5)-years old children are to be given equal chances thru Kindergarten Education.  Kindergarten would effectively promote children’s physical, social, emotional, and intellectual development, including values formation, so they will be ready for school and higher education.

The task of assessing children follows the guidelines set forth by the Department of Education (DepEd) and is in compliance with the Omnibus Guidelines on the implementation of community quarantine in the Philippines as issued by the Inter-Agency Task Force for Management of Emerging Infectious Diseases (IATF) for the school year 2020-2021, as per DepEd Order. Administration of the Philippines’ Early Childhood Development (ECD) Checklist to the learners must be before the school year starts. This will ensure that the learners are capable of meeting the expectations of the grade level. Documentation and/or certifications of previous early learning experiences, such as preschool or daycare, in addition to the result of the Philippine ECD Checklist, may be provided by the parents. The concern for the safety of both parents and teachers had to be taken into consideration depending on the Covid-19 risk severity classification of the area. In areas where physical classes are not allowed: 1) Trained parents, guardians, or adults assigned by the learner’s parent shall conduct the assessment at home, or 2) The teacher or trained community worker or volunteer may conduct the “at-home assessment based on the agreed schedules of the parents.  DepEd uses Child Record 2 of the kindergarten learners. The use of this checklist was not intended for medical diagnosis, does not determine the child’s IQ nor gauge academic achievement. Rather, it was used as a comprehensive assessment to gauge the child’s strengths and weaknesses in the different domains. Videos were created by DepEd teachers to guide the parents in administering the checklist.  The administration of the ECD Checklist by the parents may be influenced by the parent’s partiality to their child, or have an inadequate feeling in administering the checklist or burdened by it (Garbe, Ogurlu, Logand & Cook, 2020).

Gamification

Children learn best when the concepts presented are relevant to them. Teachers use games as instructional tools in their classrooms. Many games provide different sensory experiences for students. It facilitates positive emotional connections to learning as it is specifically made to be an engaging activity. Backlund and Hendrix (2013) mentioned that most games are not focused on winning and being competitive but rather on the concept of cooperation.

These learning games have been successfully designed to be used for entertainment as well as for educational purposes which have been popular over the last decade. Perez (2016) defined gamification as the application of game-design elements (a.k.a. mechanics) – such as challenges, badges, ranking and leaderboards, and storylines– in non-game contexts with the intention of modifying behaviors, increasing fidelity or engaging people, by leveraging human motivations present in games. According to Backlund and Hendrix 2013, during the game the players engage in an artificial conflict, defined by rules, that results in outcomes that are measurable. Computer-based assessment of mastery of motivation and executive functions describes such cognitive tasks. Gamification may provide a possible solution for a better assessment of EF.

Digital Trail-Making Test (dTMT)

The Trail Making Test (TMT) has been used extensively in adult neuropsychological research as an assessment of psychomotor speed, complex attention, and executive functions, as well as on children with learning disabilities and attention problems. TMT for use with children has become routine in the neuropsychological assessment of school-age children. However, the standard paper-pencil TMT was invented more than 60 years ago and has been modified into many variations. With the development of digital technologies, TMT is now modified to a digitized version (Wei, Zhao Liu & Huang, 2019).

The Digital Trail Making Test (dTMT) is a digital version of the paper-pencil Trail Making Test where the respondents “make a trail” or “connect the dots” with the speed and number of errors recorded (Yates, 2013), which was initially downloaded for the iPad App.  This first tablet prototype used a resistive touchscreen and a stylus with a force sensor which gave visual feedback to the users as they interact with the tablet (Karimpoor et al., 2017).   It was found, in a study on an analysis of a digital variant of the TMT using machine learning techniques (Dahmen, Cook, Fellows & Schmitter-Edgecombe, 2016), that the dTMT is as capable as the paper-based TMT in measuring the same aspects of cognition.

For this study, the Trail Making Test application version by Simone Sacchi, which can be downloaded for free on Google Play, was used. Sacchi mentioned in his email to this researcher that this version was developed more than 10 years ago, at a time when no digital version was yet on the market.

Executive Function of Young Children

According to Acar, Blasco, and Stanley (2016), Executive Function (EF) refers to a group of processes happening in the frontal lobe of the brain that helps control, connect, and arrange information which can be in planned behavior. EF is a broad term that includes the ability to purposefully prevent a well-learned or highly motivated but counter-productive response (inhibitory control), the ability to hold thoughts in mind while solving problems (working memory), and the ability to adjust strategies to changing contextual demands (cognitive flexibility) which provide an important foundation for learning in education settings (Zelazo, Blair, & Willoughby, 2016 as cited by Józsa, Barrett, & Morgan in 2017). There is also a correlational relationship between inhibitory control, cognitive flexibility, and/or working memory and early academic skills as well as complex learning (Schwaighofer, Bühner, and Fischer, 2017; Buttelmann and Karbach, 2017).  Blair and Raver (2015) added that EF is central to school readiness and early school achievement. The 2017 study by Benavides-Nieto, López, Quesada-Conde, and Corredor found that there was a positive relationship between young children’s social skills and executive functions. There was also an increase in the link between social competence, academic success, and executive functions.

Poowanna, et al (2022) stated that it is imperative for 21st-century learners to develop their executive function skills.  Guidelines were given to the teachers in order to facilitate the development of EF skills. These guidelines helped the teachers in creating learning materials for their students. Executive function not only boosts and improves social competence but that it increases academic achievement.

Smart Mobile Devices

Smart mobile devices refer to electronic devices which can be connected to the internet.  They are handheld therefore easy to carry around and accessible anytime. Smart mobile devices include smartphones, smart speakers, tablets, phablets (phone and tablet), and other devices which has the capabilities of a telephone, calculator, camera, video recorder, video player, music player, sketch pad, and other applications.  Wireless communications technology and the internet helps transmit information from another smart mobile device to another (Devlin, E., n.d.).

Children as early as two years old can poke, discover and learn with these mobile devices because these touchscreen devices were built for young people’s natural tendencies like touching, repeating, and trial and error (Papadakis & Kalogiannakis, 2017).

During the pandemic, education relied on mobile devices, specifically, smartphones, tablets, and laptops to foster social distancing and slow the virus’s spread. Students and parents resorted to using mobile devices in order to gain much-needed information for school work. (Ziatdinov & Cilliers, 2021). There was an increase in the use of mobile devices averaging 8-9 hours of use compared to pre-pandemic which was 6 hours. This was due to the need to accomplish tasks, socially connect, alleviate boredom, and get information about Covid-19  (Jonnatan, Seaton, Rush, Li, and Hasan, 2022).

Prior to the pandemic, teachers used to prohibit the use of mobile gadgets inside the classroom.  But now, because of the pandemic, teachers and educators also resorted to using smart mobile devices to cope with the digital transformation in education (Pozos-Pérez, Herrera-Urizar, Rivera-Vargas, and Alonso-Cano, 2022).

Gamification and Assessment of Executive Function

In many cases, assessment is confused with evaluation.  According to Yambi, 2020, assessment is the method of gathering, analyzing, and using data, for the purpose of improving current performance. Evaluation, on the other hand, is to pass judgment on the basis of the quality of a performance or work product against a standard or criteria. The proper use of assessment tools in the classroom is crucial to the success of the teaching-learning process. In order to create a positive learning experience, the proper assessments must be given so as to achieve academic success.  (Neumann, Anthony, Erazo & Neumann, 2019).  Relevant assessment tools for the digital age are available and should be utilized.  Researchers identified that it is an important initiative to create instruments for assessing school readiness and monitoring development at the beginning of schooling. Although the majority of studies on school readiness assessment have focused on the cognitive domain, recent research identified several other factors, including motivation, executive function, and emotion regulation, which play a crucial role in the preschool to kindergarten transition (Barrette, Jozsa, & Morgan, 2017).

The researcher looked into the possibility of knowing about the relative influence of cognitive performance-based executive functioning (EF) measures and behavioral EF ratings in explaining differences in children’s academic success and identifying early on the strengths and weaknesses of a child’s working memory and shifting capabilities.  This will broaden teachers’ capabilities on intervention training options to succeed in school (Dekker, Ziermans, Spruijt andSwaab, 2017). On the basis of this assessment, one can analyze the extent to which preschool EFs may be associated with academic skills such as math and literacy skills (Escolano-Pérez, Herrero-Nivela, Blanco-Villaseñor and Anguera, 2017).

There is a lot to explore about the different measures for executive functions in relation to instructional research (Schwaighofer, Bühner, and Fischer, 2017). In a brief review of the development and plasticity of cognitive flexibility across preschool to elementary school age, there was a focus on interventions that were designed to improve cognitive flexibility (Buttelmann and Karbach, 2017). Gamification is an approach in which computer games can be used to assess different cognitive processes in place of the pen-and-paper Trail-Making Test (TMT) which was known to measure executive functions tasks such as visual pattern recognition, speed of processing, working memory, and set-switching ability. In these present times, where both children and adults spend hours at a time on games, there is evidence that this may have led to changes in brain functions.

It will be a failure on the part of education to assess the educational needs of this generation while still using criteria and systems that are obsolete. Educators must anticipate and make changes to the level of content to one that is geared toward digital, virtual, and gamified modes of teaching (Jukić, and Škojo, 2021).

SUMMARY OF RELATED LITERATURE

For many years, student performance assessments have been technologically aided. These technologies replace some tasks that were previously done by teachers which allowed them to customize the tests and adopt new approaches to improve the quality of assessments. In comparison, this type of standardized assessment method takes less time and is easier to administer, and the results are readily interpretable. A well-designed, reliable, and valid test and approach to learning, especially for Executive Function skills, is needed for children in preschool and the early school grades of this generation.  A play-based assessment can provide a more detailed and reliable assessment to overcome limitations in the traditional standardized approach. The classic form of assessment consists usually of questionnaires, questions, answers, and so forth that usually interrupt and negatively affect the learning process. Incorporating games in the assessment will provide sufficient data to demonstrate the feasibility, reliability, and validity of these computer tablet-based tasks. Tablet-based or touch-screen-based assessment can fill an important need, enabling schools to assess each individual child’s level of development on each task and to develop high-quality, individualized remediation and enrichment efforts to improve the performance of our next generation of learners who are digital natives like Generation Alpha.  As to the administration of the assessment, little training is required for the test administrator since it is a connect-the-dots type of game so it is easy to give instructions and assist the child on the steps to be done in order to finish the game task.

Gamified Learning Theory

This study was grounded on the Gamified Learning Theory (GLT) which defines gamification as a process of using game-like characteristics in contexts other than gaming in order to influence learning-related behaviors or attitudes (Zaric, Roepke, Lukarov, Schroeder, 2021).  British computer programmer, and inventor, Nick Pelling first coined the word gamification in 2011. There are many variations to the definition but the focus remains.  Gamification is using game-like elements in non-gaming environments to make tasks more interesting (Landers, Auer, Collmus, & Armstrong, 2018).

According to Lumsden, Edwards, Lawrence, Coyle & Munafo (2016), game-like features can be merged with cognitive tasks to make it playful yet not diminish the scientific value of these tasks. This will help improve data quality, intervention effects, and participant engagement. It was found in their review of cognitive tasks that gamification targets executive function skills.  Their results showed that gamified tests were typically validated successfully.  It showed that although there were some mixed effects on task performance, gamified assessments, and training were highly engaging and boosted participant motivation. Gamification increases engagement with tasks that might be perceived as demotivating. They also found that gamification showed encouraged self-motivation, improving enjoyment and even reducing anxiety.

In a study involving the Trail Making Test in screening executive functions, Wong, Wu, and Tu (2022) found that gamification showed a significant enhancement after training, in both executive function skills as well as in attention span.

Educators found difficulty and hope in the possibility of solutions to motivate students through these technological advances (Dichev, Dicheva, Angelova, & Agre, 2015). Whether it be on a cell phone, tablet, or PC, students are more able to access information and can do it more quickly than ever before (Dichev et al., 2015). Children are by nature playful and technology changed children’s engagements which made the gathering of information and knowledge fast (Dichev et al., 2015).  Although GLT implies that gamification does not directly affect learning it is, however, used to stimulate learning-related behavior in a mediating or moderating process which happens when the designer intends to encourage behavior or attitude that will improve learning (Zaric, et, al. 2021).

Online gaming, mobile phone games, and home console games all have the capacity to engage students for hours. In education, educators’ found it a useful tool to motivate and for students to learn.  Students engaged in a game show cognitive attitudes.  They show focus yet remain playful and spontaneous while they are in the game. Fengfeng Ke, Kui Xie, and Ying Xie (2015) suggested that for game-based learning to be an effective engagement, it has to be combined with the challenges.  It has to be uninterrupted engagement while situated in a playful situation.  Play-strategy reflection is a conscious, meta-reflective process that helps to advance or transform general cognitive engagement into a content engagement. But this reflective engagement is the last thing players think of when in play-based cognitive activities.

In other words, the game stimulates the player to do the tasks which assess their cognitive abilities in a reflective way since the player will think well first before an action is done. This was shown in the Trail Making Test which has a game-like design with the simple task of pressing buttons in consecutive order in the fastest time possible (numbers 1-25 in Part A and 1-13 alternate with letters A-L in Part B) (Carlson & Vusak, 2020).

Also, the developmental perspective of Executive function (EF) proposes the foundational components of executive function which are working memory, flexible thinking, and self-control. EF happens during the early years of life and continues to intensify as children reach childhood and adolescence. According to Barkley (2011), EF is a self-directed set of actions that are intended to alter a delayed future outcome, and serves as an umbrella term to include goal-oriented control functions of the prefrontal cortex.

Attention ability and executive function will gradually decline as individuals age (Wong, Wu & Tu, 2022). It is important to know and understand executive functions and how changing techniques in educational programs can help students’ performance aid in their future success in their development and in academic life.

Statement of the Problem

The purpose of this quantitative experimental study was to examine the use of gamification techniques in the assessment of executive function of kindergarten children at Concepcion Integrated School.

This study sought to answer the following questions:

1. What is the level of executive function of the pupils before and after exposure to the gamification technique?
2. Is there a significant difference in the level of executive function before exposure to gamification and after gamification?
3. Is there a significant difference between the executive function of the pupils who used numbers only and numbers and letters in the gamification technique?
4. What intervention training program can be proposed based on the results of the study?

Hypotheses

H01 There is no significant difference in the level of executive function of the pupils before and after exposure to gamification.

H02 There is no significant difference between the executive function of the pupils who used numbers only and numbers and letters in the gamification technique.

SCOPE AND DELIMITATION

               The researcher used a new device-aided cognitive function test, Trail Making Test by Simone Sacchi (TMT), a digital version of the paper pencil test Trail Making Test to assess EF skills. To distinguish the application for this study, the researcher added “digital” before the Trail Making Test (dTMT) to distinguish the app from the paper pencil test. The app is free to download on any brand of android  smartphone.  For IOS smartphones, a similar application may be downloaded but it is named as Trail Making Test Lite J.  For this particular study, the Android version was used since all the participants used Android smartphones.

               All the ten (10) kindergarten pupils who regularly attended the online class at Concepcion Integrated School tried the gamification app with the help of their parents. Because of the pandemic, only the pupils who regularly attended the online classes were selected.

METHODOLOGY

This chapter presents the description of the research method used in the study. It includes research design, research method, research instrument, and population and sample.

Research Design

This is a One-group Pretest-posttest research design which is quantitative experimental research. The researcher used one group to gather information and examine the use of the gamification technique in the assessment of the executive function of kindergarten pupils.  According to Gregory A. Cranmer in The SAGE Encyclopedia of Communication Research Methods, a one-group pretest–posttest design is a type of research design that is most often used by behavioral researchers to determine the treatment or intervention on a given sample from the outcome of the test. This research design has two characteristics. The first is the use of a single group of participants (i.e., a one-group design) which means that all participants are part of a single condition—meaning all participants are given the same treatments and assessments. The second feature is to assess the dependent variable before and after treatment (i.e., a pretest–posttest design).

There are two ways in which the digital TMT to interact with the screen. First, the sequence is selected by simply touching the dots on display and a straight line will appear to link the selected dots, green if the selected dot is correct and red if the sequence is incorrect.  A red line will be created if the answer is incorrect. to delete the red line to try again, the user simply clicks on the background. 2) The second modality is activated by selecting FINGER PAINT in the settings menu,  directly drawing a line on the touchscreen to connect to the next sequence of circles.

For this study, the pupils used the first modality where they simply touched the selected circles or dots to complete the sequence. The entire procedure consists of the sample with 8 circles followed by a full trial, for both Part A (circles 1 to 25) and Part B (circles 1 to 13, with numbers and letters). At the end of the procedure, the parents can view the participant’s name, which they typed in before the start of the game, and the time that indicated how long it took for the participant to finish the test. The parents of the pupils took a screenshot of their child’s results and sent it to the cooperating teacher who then sent it to this researcher via messenger.  A pre-test and a post-test were then conducted.

The results of the pre-test were then post-test and then compared.  The results of Part A, which was a numbers sequence, were also compared to Part B, which was composed of numbers and letters in alternate sequences.

Locale of the study

This study was conducted online via Zoom and messenger with the assistance and participation of a kindergarten teacher at Concepcion Integrated School (CIS), a public educational institution in Concepcion Uno, Marikina City. It is considered the only integrated school in the city. This humble institution of learning provides both elementary and secondary education to young Marikenos. In addition to having a high-quality basic education program, the school has also been active in conducting various school projects and programs beneficial to the entire school community. It also continues to produce competent graduates equipped with relevant skills and knowledge essential to thrive in their chosen fields.

Concepcion Integrated School is duly recognized by the Department of Education (DepEd) as well as the Schools Division Office (SDO) of Marikina City.

Participants
Ten (10) kindergarten pupils in an online class during the school year 2020-2021 and their parents were the participants and respondents of this study. Since the Covid-19 pandemic, classes were shifted online. These pupils were chosen because they were the only ones who regularly attended online classes of the 30 students in that class. The rest opted for modular distance learning.

Instruments of the Study

Digital Trail-Making Test (dTMT)

               For the purpose of this study, the researcher used the Digital Trail Making Test by Simone Sacchi application for smartphone Android devices.  This is the same neuropsychological test called the “Trail Making Test”, only this is in digital form. The application can be downloaded for free via Google Play Store for Android smartphones. For IOS,  the Trail Making Test J Lite is available for free via the App Store.  There is, however, a difference in the face feature of the app- the IOS has a gray background with just numbers and or letters- no circles or disks on display.  Similar to connect-the-dots, the sequence is selected by simply touching the circles on display, then straight lines will appear that connect the selected circles, green or red depending on whether the sequence was correct or not. If a red line appears, it means the selected circle is wrong. To delete the red line, the participant clicks on the black background and then proceeds to select the next circle.  The entire procedure consists of the sample with 8 circles followed by a full trial, for both Part A (circles 1 to 25) and Part B (circles 1 to 13, with numbers and letters: 1-A, 2-B, 3-C… 13).  At the end of the procedure, the participant then clicks on a dot to view the participant’s name and the timing of the tests.

The table below shows the scoring scale for the Trail Making Test.

Data Gathering Procedure

The researcher first corresponded with Ms. Mary Laine Eriarte in June of 2021 about the possibility of having her kindergarten class participate in this assessment trial as well as a resource person regarding the study.  The researcher informed and demonstrated the use of the Trail Making Test for Android smartphones by Simone Sacchi.  However, since the pandemic hit, it was difficult for teachers, parents, and students to coordinate and correspond. It was a challenge considering that the country underwent a health and economic crisis. Covid19 physical and mental health concerns put the study on hold. In 2021, the parents, teachers, and students were mostly at ease with online classes, downloading files, answering Google forms, and using applications that the teacher used. This became timely for the researcher to continue with the study but to great lengths to follow health protocols.

A message from the cooperating teacher was sent to the parents of the pupils who regularly attended the online class.  A Zoom meeting was set to get to meet the parents, get to know the pupils, and introduce the application. Instructions were given on how to download and administer the Trail Making Test. They were also instructed to answer a link to a Google Form to find out the parent’s name, age, and gender and their child’s name, birthday, and gender.  The researcher also thought, in consultation with the teacher, it was best to give a two hundred pesos (Php 200) token or load for data to the parents to access the internet in order to download and answer the form.  Also, since it will be a pre-test and post-test task, the researcher did not want to burden the parents and deter anyone from answering the test.

The parents took a screenshot of their child’s timestamp/results on both Parts A and B of the dTMT. This was sent to the teacher through FB messenger who then sent it to this researcher.  After a month, the parents were again contacted by the teacher to administer a post-test. In the same manner, screenshots of the results were sent through the teacher who then sent these screenshots to the researcher. The researcher then analyzed the data given.

The pre-test was taken on August 23, 2022 and the post-test was conducted on September 20, 2021. Screenshots of Google meet, and dTMT results are attached in the appendices.

DATA ANALYSIS

This study utilized the experimental research using quantitative method with the following statistical tools:

For problem number 1, to determine the level of pupils’ executive function in the pretest and posttest using gamification technique

For problem number 2, to determine if there is a significant difference between the pupils’ level of executive function in the pre-test and post-test using gamification technique.

For problem number 3, to determine if there is a significant difference in the numbers only (Part A) and in numbers and letters (Past B) of the digital Trail Making Test.

For problem number 4, to propose an intervention training program based on the results of the study.

Ethical Considerations

Due to the pandemic, the education system shifted to online classes. It was difficult to gauge whether it was safe to go out or to meet with the teachers and school administrators. Announcements of no classes were randomly declared depending on the rise of Covid patients in the area. The researcher tried to talk to anyone at the school to seek permission to conduct the study. But because of the health crisis, classes were often disrupted indefinitely. It was during these uncertain times that the researcher reached out and was able to communicate directly with Ms. Eriarte through messenger.  Parents had to use smartphones so that their children can attend online classes and study using their gadgets.  Parents also had to pay for their load for internet use or data. In this regard, the researcher, in consultation with the cooperating teacher, sent 2000 pesos (200 for each of the 10 parents), through the teacher’s GCash account, in order for the parents to have access to the internet, download the app, administer and use the app, as well as attend online meetings, so as to not add to their burden for load or data consumption.

The meeting with the parents also happened after the moving-up exercises, out of respect to the request of the teacher as they had to attend to parents and students and had to finish other school tasks like examinations, paperwork, and other requirements for the moving-up ceremonies- the end of the school year 2020-2021, while still in the midst of the pandemic.

The researcher also corresponded, through Google and via email, with the Trail Making Test maker, Simone Sacchi.  Google Support Team’s Micheal J. replied and advised this researcher to contact the developer directly. Fortunately, Simone Sacchi responded on October 5, 2022, and answered some of the questions this researcher had.

RESULTS

The data gathered through the use of the different instruments are hereby presented in this chapter. Analysis of data was made with appropriate interpretations and inferences so that conclusions may be drawn.

Problem 1: Level of executive function of the pupils in the pre-test and post-test using the gamification technique   

To determine the level of executive function of the pupils, the digital Trail Making Test by Simone Sacchi was used.  The test has both Part A, where numbers only were considered and Part B, where both numbers and letters, and the mean of the data was computed. Table B scale below was used:

Table B. Trail Making Test Scale Part A and B

Table 1.1. Level of Executive Function of the Pupils’ Pre-test Using Gamification Technique

Table 1.2. Level of Executive Function of the Pupils’ Post-test Using Gamification Technique

Problem 2. Significance of difference in the level of executive functions of the pupils in the pretest and posttest using the Gamification technique

To affirm the significance of difference in the level of executive function of the pupils in the pretest and posttest the data were subjected to paired t-test

Table 2. Difference in the Level of Executive Function of the Pupils in the Pretest and Posttest Using the Gamification Technique

Note: p-value ≤ 0.05 – significant, p-value > 0.05 – not significant

Problem 3. Significant difference between the executive level of pupils who used numbers only and numbers and letters gamification technique.

To verify the significance of the difference between the executive level of pupils who used numbers only and numbers and letters in the gamification technique, the data was put through paired t-tests.

Table 3. Difference Between the Executive Level of the Pupils in Numbers Only and Numbers and Letters Gamification Technique

Note: p-value ≤ 0.05 – significant, p-value > 0.05 – not significant

Problem 4. Proposed Intervention Training Program

Suggested below are some simple strategies that every teacher can do to help all learners develop executive functioning successfully in the classroom:

Table 4. Proposed Strategies to Help Learners Develop Executive Function Skills

Most teachers already practice these activities every day in their classrooms but are not aware of its significance to the development of executive functioning skills. Hopefully, these strategies will aid the teachers in helping their pupils.

DISCUSSION

Problem 1: Level of executive function of the pupils in the pre-test and post-test using the gamification technique   

As shown in Table 1.1 the pupils’ have an average level of executive function for both Part A (Numbers only) and Part B (Numbers and letters) before exposure to the gamification technique as revealed by the mean of 01:10 or 1 minute and 10 seconds and 02:37 or 2 minutes and 37 seconds time respectively. As indicated in the table above for part A with the use of numbers only and for part B where both numbers and letters were in the test, most of the pupils finished the test within 29 -77 seconds or 29 seconds.

For Part A, six (6) pupils had a description of average while four (4) pupils were classified as deficient.  For Part B, one (1) pupil was deficient and took longer to finish the test at 6 minutes and 30 seconds. The pre-test scores in the gamification approach in Part A were faster than in Part B. This indicates that Part A, where only numbers were sequenced, was easier to accomplish than Part B which was the numbers and letters sequence.  Part A of the dTMT requires visual scanning, number sequencing, and motor speed. Part B requires more skills  which are visual scanning, number sequencing, letter sequencing, number and letter switching, and motor speed.  Part B requires cognitive flexibility which is a higher-order executive functioning which was why pupils experienced some difficulty and took longer in finishing the task (Hopkins, 2019).

Table 1.2 reflected the level of executive functions of the pupils after exposure to gamification techniques. For Part A, using numbers only, the level of executive function of the pupil is average with mean score of 01:00 or 1 minute of time in doing the activity.

Of the ten (10) pupils, 2 were deficient, 2 were advance and 6 were average.

For Part B, using numbers and letters, the pupils’ level of executive function had a mean score of 02:16 or 2 minutes and 16 seconds which was average in the scale. Moreover, of the ten (10) pupils, 5 were advance, 4 were average and 1 was deficient. Note that the same pupil who got a deficient score in both Part A and Part B.  As shown in the table above, most of the pupils who completed the trail making test in both parts A (numbers only) and B (numbers and letters) within 75 – 272 seconds or 1 min, 25 seconds – 4 minutes and 53 seconds categorized as average.

The pupils’ time was lesser meaning they were able to finish the test faster in the post-test. This indicates that the pupils have become familiar with the new knowledge and were able to give attention and improvements which were relevant in the post-test (Korzeniowski, Gabriela, Greco & Monteliva, 2020).

Problem 2. Significance of difference in the level of executive functions of the pupils in the pretest and posttest using the Gamification technique

As shown in Table 2, the computed p-value of 0.03 and 0.01 on the difference between the level of the executive function of the pupils in the Gamification technique using numbers only and numbers and letters before and after exposure to the trail-making test are both less than 0.05. Hence, the null hypotheses are rejected. This means that  significant difference exists between the Pre-test and the Post-test using the gamification technique.

Most students were able to finish the test at a faster rate in the post-test than in the pre-test even though the scores both fall under the average category. This indicates familiarity with how the test goes.

According to Stratton S. (2019), the use of a quasi-experiment pre-test and post-test has its advantage and disadvantage. It is good that the variable is being tested before and after an intervention is given. But that associations from outcomes are made especially if the participants are not randomly selected. Stratton added that pre-testing allows the participants to be sensitized and inform the other participants of their familiarity with the test, and share their experiences on how the test was taken or how easy it is to do the test. This could allow others to score higher in the post-test.  This may be reflected in individual scores where some students were fast enough to change categories from deficient to average or from average to advance.

Fengfeng Ke, Kui Xie, and Ying Xie (2015) suggested that game-based learning engagement is an integrated and continuous process that advances from affective engagement driven by optimal challenge, cognitive engagement situated in playfulness, to potentially game- action-based content engagement. In other words, the game stimulates the player to do the tasks which assess their cognitive abilities in a reflective way since the player will think well first before an action is done.

It should also be noted that using technology effectively is key to educational design for these students who belong to Generation Alpha. In essence, educators need to become more assessment literate, and professional practices regarding assessment and evaluation have witnessed a recognizable change in the last few decades (Yurtseven & Karadeniz, 2020).

Problem 3. Significant difference between the executive level of pupils who used numbers only and numbers and letters  gamification technique.

As reflected in Table 3, the computed p-value of 0.07 for the difference from the pretest to post-test in the pupils’ level of executive function using the Gamification technique for Part A, numbers only, exceeded 0.05 which leads to a decision of not rejecting the null hypothesis, hence the difference from the pretest to the posttest using the gamification for numbers only is not significant.

For the difference from the pretest to the post-test of the level of executive function of the pupils using the Gamification technique for Part B, Letters, and Numbers, the computed p-value 0.03 is less than 0.05 suggesting significant difference between the pretest and posttest of the pupils in the gamification technique using numbers and letters.

This indicates that the level of executive function of the pupils can be attributed to 1) the trail-making test has been taken by the pupils. They have become familiar with the game; 2) when taking Part B of TMT, children have become more familiar with numbers than with letters since Part A is numbers only; 3) Part B of the test is more complex than Part A. Cognitive flexibility is needed in order to perform the shifting task in Part B. Cognitive flexibility is activated by the prefrontal lobe which is part of the brain that activates the higher order executive functioning (Smith-Watts, Ahern, Jones, Farrer, and Correia, 2019).

According to Shubert (2016), digits or numbers and letters are fairly similar in form and, as visual stimuli, identification processes are shared between the two character types as backed by evidence from normal and impaired readers supporting the similarity of digit and letter identification. It has also been noted that digit identification is often more accurate than letter identification, and many authors have proposed possible explanations for this phenomenon (Moura, Haase, Lopes-Silva, Batista, de Freitas, Bahnmueller & Moeller in 2021).

Moura, et al 2021 noted that digit identification is often more accurate than letter identification, and many authors have proposed possible explanations for this phenomenon.

Connectionism by Thorndike (1898) stated bonds or associations need to be formed in order for learning to take place.  For an agreeable result to happen, there has to connection with that particular certain situation. His work provided that associations had to be made in order for learning to take place.   It was also suggested that learning of both verbal and numerical codes proceeds from initial sequential processing in the working memory resources which later becomes more holistic and thus processing becomes more automatic.

Problem 4. Proposed Intervention Training Program

It is important to 1) teach EF skills to students because: a) it teaches children how to learn; b) it empowers students to take control over their learning; c) promotes motivation, focused effort and hard work; d) increases self- confidence and encourage independence and helps students understand their learning profile (Meltzer, 2018; Meltzer and Basho, 2010).

More valuable is that teachers will be be empowered with the knowledge about what EF is, what EF skills are and how they can prepare their lessons and create classroom environment that will help pupils in the development of their EF skills.

First, use the Trail Making Test App.  The app is easy to download, free of charge and easy to use. In the initial use of the app at the beginning of the school year, the teacher will be made aware of the possible difficulty or challenges that the child will encounter.  After knowing their pupils’ initial assessment, teachers can provide immediate and appropriate lessons and activities as intervention in relation to the results. A post test can be conducted every quarter to assess improvements in the executive functioning of the pupils. Zeng, Miao, Leung, & Shen, 2017 suggested that the computerized TMT is suitable for long-term observation of cognitive skills. This will somehow give the teacher a comparison of the pupil’s performance with the initial test and see if the intervention worked. With the results on hand, together with observations and monitoring,  results of the intervention and post test, the teacher can then recommend to the school and the parents if the pupil needs to be assessed further by a psychometrician.

Next, teachers, parents and guardians should be given a seminar about what executive function is all about and how knowledge about this can help them provide better for their students. At this seminar or more aptly, a webinar, a copy of the presentation will be emailed to them or a link will be sent in order for them to download this material free of charge. This material will be about executive function, including definitions, skills and strategies in order to educate them, be conscious of their and their students’ actions and behaviors, as well as strategies on how to change behaviors appropriately. The webinar could last for an hour and would possibly cost less than Five thousand pesos (Php 5,000.00) depending on the honorarium for the speaker/s. Some speakers would do it free of charge as part of their advocacy. E-certificates will be given to participants after responding to a feedback form. Questions from participants may be posted in the chat box and answered by the speaker/s. This webinar will be free of charge to the participants.

A live seminar or conference will cost more as it would entail expenditures for the venue, audio-visual systems, snacks for the speakers and participants, materials for conference and certificates, and honorarium for the speakers. This could cost less if the venue will be held in a school AVR or gym as it would accommodate more people but ideally there should be 50 to 100 participants. Some charge from 500-5,000 pesos per participant depending on the venue.

SUMMARY OF FINDINGS

This paper attempted to give insights into the use of an assessment tool in the context of educational assessment by assessing the executive function of kindergarten pupils using gamification in smart mobile devices, specifically the Trail Making Test by Simone Sacchi.  

Among the important findings of this research were:

1. Level of executive function of the pupils before and after exposure to the gamification technique.

The pupils’ have an average level of executive function for both Part A (Numbers only) and Part B (Numbers and letters) in the pre-test using gamification technique as revealed by the mean of 1.10 or 1 minute and 10 seconds and 2.37 or 2 minutes and 37 seconds time respectively and are both average. On the other hand, the level of executive functions of the pupils in the post-test for part A or using numbers only is average as evidenced by 1.00 or 1 minute time in doing the activity. Likewise, the students’ level of executive function for part B or using numbers and letters in the gamification technique is also average as described by the mean of 2.16 or 2 minutes and 16 seconds.

2. Significance of difference in the level of executive functions of the pupils in the pretest and posttest  using gamification technique.

The computed p-value of 0.03 and 0.01 on the difference between the level of the executive function of the pupils in the pre-test and post-test using gamification technique using numbers only and numbers and letters before and after exposure to the trail making test are both less than 0.05.

3. Significant difference between the executive level of pupils who used numbers only and numbers and letters gamification technique.

The computed p- value of 0.07 for the difference from the pretest to post test in the pupils’ level of executive function using the Gamification technique for Numbers only exceeded 0.05 and therefore significant.

For the difference from pretest to posttest of the level of executive function of the pupils using Gamification technique for Numbers and Letters, the computed p-value 0.03 which is less than 0.05 and therefore significant.

CONCLUSIONS

Based on the foregoing findings, the following conclusions were drawn:

1. The time to finish the Trail Making Test is shorter therefore faster for the gamification technique with both numbers only and the numbers and letters in the pre-test and post test.
2. Significant difference occurred in the pre-test and post-test using the Trail Making Test for both numbers only and numbers and letters, therefore the null hypothesis was rejected.
3. No significant difference occurred between the pre-test and post-test using the numbers only. Therefore the null hypothesis was not rejected. But significant difference occurred between the gamification technique for numbers and letters hence, the null hypothesis for was rejected. This means that using gamification technique that incorporates the numbers and letters in assessing the executive functions of the kindergarten pupils is more effective than in using numbers only..

RECOMMENDATIONS

In the light of the conclusions drawn from the study, the following are hereby recommended:

1. Focus on the integration and enhancement of incorporating numbers only and both numbers and letters in the use of gamification technique since a difference occurred between their use.
2. Gamification technique yielded a positive result therefore using the app is highly recommended specifically in an online learning modality where pupils need not go to the school physically for assessment purposes, hence, reduces the risk of contracting covid-19 virus.
3. It is recommended that more testing of this app be done on a larger population or large-scale educational study. The app should be tried also by other pupils both in the private and public schools.
4. Success in the use of gamification app will depend upon close collaboration between educators, students, and policy makers in the design, development, and utilization of technology-based assessments.
5. It is with the hope that this type of testing be done before classes begin as part of the assessment for admission to kindergarten. An initial posttest should be done a month after classes have started. After results are seen, teachers may be able to plan a proper course of action as to draft lessons and activities on how these executive functions be addressed so that the student can cope in class. The trail making test assessment may be repeated after intervention so as  give recommendations to parents, administration and/or to other professionals.
6. Use of variables other than numbers and letters in the gamification technique can be studied and tried out.

REFERENCES

1. Backlund P., & Hendrix, M. (2013) Educational games – are they worth the effort? A literature survey of the effectiveness of serious games. Conference Paper. September 2013 DOI: 10.1109/VS-GAMES.2013.6624226 Research Gate https://www.researchgate.net/publication/261076302 Retrieved September 2018.
2. Barrett, K. C., Józsa, K., & Morgan, G. A. (2017). New computer-based mastery motivation and executive function tasks for school readiness and school success in 3 to 8 year-old children. Hungarian Educational Research Journal, 7 (2), 89–108. doi:10.14413/h34j.2017.02.06
3. Benavides-Nieto, A., López, M., Quesada-Conde, A. and Corredor, G. (2017). Basic executive functions in early childhood education and their relationship with social competence. February 2017 Procedia – Social and Behavioral Sciences 237:471-478 DOI: 10.1016/j.sbspro.2017.02.092 https://www.researchgate.net/publication/314714506 Basic Executive Function in Early Childhood Education and their Relationship with Social Competece
4. Buttelmann,F., & Karbach, J. (2017). Development and plasticity of cognitive flexibility in early and middle childhood . Frontiers in Psychology, published: 20 June 2017 doi: 10.3389/fpsyg.2017.01040. Retrieved August 2018 from https://www.frontiersin.org/articles/10.3389/fpsyg.2017.01040/full
5. Cranmer, G. A. (2022). One-group pretest–posttest design in The SAGE Encyclopedia of Communication Research Methods. Retrieved July 2, 2022 from https://methods.sagepub.com/reference/the-sage-encyclopedia-of-communication-research-methods/i9778.xml
6. Dahmen, J., Cook, D., Fellows, R., & Schmitter-Edgecombe, M. (2017).  An analysis of a digital variant of the trail making test using machine learning techniques. Technol Health Care. 2017; 25(2): 251–264. doi:10.3233/THC-161274. Retrieved June 6, 2022 from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5384876/pdf/nihms845821.pdf
7. Dekker, M.C., Ziermans, T.B., Spruijt, A.M. & Swaab, H. (2017) Cognitive, parents and teacher ting measures of executive functioning: Shared and unique influences on school achievement. Frontiers in Psychology published: 30 January 2017 doi: 10.3389/fpsyg.2017.00048. Retrieved August15, 2018 from https://www.frontiersin.org/articles/10.3389/fpsyg.2017.00048/full
8. Department of Education Revised Philippines Early Childhood Development Checklist for 3.1-5.11 years old Technical and Administration Manual for Teachers. (2021). Retrieved December 20, 2021 from https://depedtambayan.net/philippine-early-childhood-development-checklist/
9. Diamond, A. (2013). Executive functions. Annu. Rev. Psychol. 64, 135–168. doi: 10.1146/annurev-psych-113011-143750 (In Nieto, M., Ros, L., Medina, G., Ricarte, J.J., Latorre, J.M. (2016). Assessing executive functions in preschoolers using shape school tasks. (Frontiers in Psychology journal, published: 27 September 2016 doi: 10.3389/fpsyg.2016.01489)
10. Fengfeng Ke, Kui Xie and Ying Xie (2015). Game-based learning engagement: A theory- and data-driven exploration © 2015 British Educational Research Association; British Journal of Educational Technology (2015) doi:10.1111/bjet.12314 . https://www.researchgate.net/publication/280916567 Retrieved August 2018
11. Fellows, R., Dahmen, , J., Cook, D. and Schmitter-Edgecombe, M. (2017) Multicomponent analysis of a digital Trail Making Test.  Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5286906/
12. Fulton, J. (2019). Theory of gamification-motivation. Retrieved June 19, 2022 from https://files.eric.ed.gov/fulltext/ED607091.pdf
13. Garbe, A., Ogurlu, U., Logan, N., & Cook, P. (2020). COVID-19 and remote learning: experiences of parents with children during the pandemic. American Journal of Qualitative Research December 2020, Vol. 4 No. 3, pp. 45-65 https://doi.org/10.29333/ajqr/8471 © 2020 AJQR. http://www.ajqr.org as retrieved January 2022 from https://www.ajqr.org/download/parents-experiences-with-remote-education-during-covid-19-school-closures-8471.pdf
14. Howard, S. J., Okely, A. D. (2014) Catching fish and avoiding sharks: Investigating factors that influence developmentally appropriate measurement of preschoolers’ inhibitory control. Journal of Psychoeducational Assessment 2015, Vol. 33(6) 585–596 © 2014 SAGE Publication; sagepub.com/journalsPermissions.nav DOI: 10.1177/0734282914562933 Retrieved September 2018
15. Howard, S. J., Okely, A. D., and Ellis, Y. G. (2015). Evaluation of a differentiation model of preschoolers’ executive functions. Front. Psychol. 6:285. doi: 10.3389/fpsyg.2015.00285 (In Nieto, M., Ros, L., Medina, G., Ricarte, J.J., Latorre, J.M. (2016). Assessing executive functions in preschoolers using shape school task. (Frontiers in Psychology journal, published: 27 September 2016 doi: 10.3389/fpsyg.2016.01489)  https://www.frontiersin.org/articles/10.3389/fpsyg.2015.00285/full
16. Jonnatan, L., Seaton, CL., Rush, KL., Li, EPH., and Hasan, K. (2022) Mobile device usage before and during the covid-19 pandemic among rural and urban adults. Int J Environ Res Public Health. 2022 Jul 6;19(14):8231. doi: 10.3390/ijerph19148231. PMID: 35886082; PMCID: PMC9315523. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9315523/
17. Jozsa, K., Morgan, G.A. & Barrett, K. (2017). Game-like tablet assessment of approaches to learning: Assessing mastery motivation and executive functions. Electronic Journal of Research in Educational Psychology. December 2017 DOI: 10.14204/ejrep.43.17026 · Retrieved July 2018 from https://www.researchgate.net/publication/323170393 Game-like Tablet Assessment of Approaches to Learning Assessing Mastery Motivation and Executive Functions
18. Jukić, R. and Škojo, T. (2021).The educational needs of the alpha generation. 44th International Convention on Information, Communication and Electronic Technology (MIPRO), Opatija, Croatia, 2021, pp. 564-569, doi: 10.23919/MIPRO52101.2021.9597106. https://ieeexplore.ieee.org/document/9597106/authors#authors
19. Karimpoor, M., Churchill, N., Tam, F., Fischer, C., Schweizer, T. & Graham, S. (2017). Tablet-based functional mri of the trail making test: Effect of tablet interaction mode. In Frontiers in Human Neuroscience, 24 October 2017 | https://doi.org/10.3389/fnhum.2017.00496 Retrieved December, 22, 2021 from https://www.frontiersin.org/articles/10.3389/fnhum.2017.00496/full
20. Korzeniowski, C.. Gabriela, M. Greco, C. & Monteoliva, J. M. (2020). Improving executive functions in elementary schoolchildren. European Journal of Psychology and Educational Research, 3(1), 59-73. DOI:10.12973/ejper.3.1.59. Retreived July 15, 2022 from https://www.researchgate.net/publication/342461220 Improving Executive Functions in Elementary Schoolchildren#pf6
21. Landers, R. N., Auer, E. M., Collmus, A. B., Armstrong, M. B. (2018). Gamification science, its history and future: Definitions and a research agenda. Simulation & Gaming, 49, 315-337. In Gamification: Concepts, Consequences and Critiques by Mike Vesa and J. Tuomas Harviainen, Journal of Management Inquiry. Sage Journals. Retrieved July 20, 2022 from https://journals.sagepub.com/doi/full/10.1177/1056492618790911
22. Lumsden, J., Edward, E.A., Lawrence, N. S., Coyle, D. &Munafò, M.R. (2016) Gamification of cognitive assessment and cognitive training: A systematic review of applications and efficacy. JMIR Publications Downloaded 2018-08-23T07:03:11Z Retrieved August 2018 from http://dx.doi.org/10.2196/games.5888  https://games.jmir.org/2016/2/e11/
23. Moura, R. Haase, V.G., Lopes-Silva, J.B., Batista, L., de Freitas, F., Bahnmueller, J. and Moeller, K. (2021). Reading and writing words and numbers: Similarities, differences, and implications. Retrieved August 3, 2022 from https://www.researchgate.net/publication/351886992 Reading and writing words and numbers Similarities differences and implications
24. Neumann, M., Anthony, J., Erazo, N., & Neumann, D. (2019) Assessment and technology: Mapping future directions in the early childhood classroom.  Front. Educ., 18 October 2019 | https://doi.org/10.3389/feduc.2019.00116 Retrieved June 2, 2022 from https://www.frontiersin.org/articles/10.3389/feduc.2019.00116/full
25. Nieto, M., Ros, L., Medina, G., Ricarte, J.J., & Latorre, J.M. (2016). Assessing executive functions in preschoolers using shape school task. (Frontiers in Psychology journal, published: 27 September 2016 doi: 10.3389/fpsyg.2016.01489) Retrieved August 2018 from https://www.frontiersin.org/articles/10.3389/fpsyg.2016.01489/full
26. Papadakis, S. & Kalogiannakis, M. (2017) Mobile educational applications for children: What educators and parents need to know January 2017 International Journal of Mobile Learning and Organisation 11(3):256 DOI:10.1504/IJMLO.2017.085338 https://www.researchgate.net/publication/318665186 Mobile educational applications for children What educators and parents need to know
27. Poowanna, B., Sarnkong, R., Wangsitthidet, S., Srikula, W. and Nakunsong, T. (2022). The development executive functions for early childhood in 21st century. Journal of Education and Learning; Vol. 11, No. 4; 2022 ISSN 1927-5250 E-ISSN 1927-5269 Published by CanadianCenter of Science and Education. https://files.eric.ed.gov/fulltext/EJ1346129.pdf
28. Pozos-Pérez, K., Herrera-Urizar, G., Rivera-Vargas, P., and Alonso-Cano, C. (2022) Use of mobile phones in classrooms and digitalisation of educational centres in barcelona. Educ. Sci. 2023, 13, 21. https://doi.org/10.3390/educsci13010021
29. Schwaighofer, M., Buhner, M. & Fischer, F. (2017) Executive functions in the context of complex learning: Malleable moderators? Frontline Learning Research Vol.5 No. 1 (2017) 58 – 75 ISSN 2295-3159. Retrieved August 2018 from https://eric.ed.gov/?id=EJ1138175
30. Simfukwe, C., Youn, Y.C., Kim, S.Y. & Soo An, S. (2021) Digital trail making test-black and white: Normal vs MCI. In Taylor & Francis Online Retrieved June 2, 2022 from https://www.tandfonline.com/doi/abs/10.1080/23279095.2021.1871615
31. Skogan, A., Zeiner, P., Egeland, J., Urnes, A., Reichborn-Kjennerud, T. and Aase, H. (2015). Parent ratings of executive function in young preschool children with symptoms of attention-deficit/-hyperactivity disorder. https://behavioralandbrainfunctions.biomedcentral.com/track/pdf/10.1186/s12993-015-0060-1.pdf
32. Smith-Watts, A., Ahern, D., Jones, J., Farrer, T and Correia, S. (2019) Trail-making test part b: Evaluation of the efficiency score for assessing floor-level change in veterans. Archives of Clinical Neuropsychology, Volume 34, Issue 2, March 2019, Pages 243–253, https://doi.org/10.1093/arclin/acy025
33. Wei, W., Zhao, H., Liu, Y., & Huang, Y. (2019). Traditional trail making test modified into brand-new assessment tools: Digital and walking trail making test. In Research Gate November 2019 Journal of Visualized Experiments 2019 (153) DOI: 10.3791/60456 Retrieved May 22, 2022 from https://www.researchgate.net/publication/337475327 Traditional Trail Making Test Modified into Brand-new Assessment Tools Digital and Walking Trail Making Test/stats
34. Wong, Y.-K., Wu, C.-F., and Tu, Y.-H. (2022).  Effectiveness of a serious game design and game mechanic factors for attention and executive function improvement in the elderly: A pretest-posttest study. Appl. Sci. 2022, 12, 6923. https://doi.org/10.3390/app12146923
35. Yambi, T. (2018). Assessment and evaluation in education. Researchgate. Retrieved June 5, 2022 from https://www.researchgate.net/publication/342918149 ASSESSMENT AND EVALUATION IN EDUCATION
36. Yurtseven, N. & Karaadeniz, S. (2020). The teacher of generation alpha. chapter 1 in The teacher of generation alpha Peter Lang Publishing. Retrieved June 13, 2022 from https://www.researchgate.net/publication/349180864 The Teacher of Generation Alpha Chapter 1
37. Zaric, N., Roepkel R., Lukarov, ., & Schroeder, U. (2021) Gamified learning theory: The moderating role of learners’ learning tendencies. International Journal of Serious Games, Volume 8, Issue 3,September 2021 ISSN: 2384-8766 http://dx.doi.org/10.17083/ijsg.v8i3.438  https://journal.seriousgamessociety.org/index.php/IJSG/article/view/438/430
38. Zelazo, P. D., Blair, C. B., & Willoughby, M. T. (2016). Executive function: Implications for education (NCER 2017–2000) . Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education https://files.eric.ed.gov/fulltext/ED570880.pdf
39. Ziatdinov, R. & Cilliers, E. (2021). Generation alpha: Understanding the next cohort of university students October 2021 DOI:10.13187/ejced.2021.3.783 Retrieved September 15, 2022 from https://www.researchgate.net/publication/355146275 Generation Alpha Understanding the Next Cohort of University Students