Enhancing Road Safety with AI: A Multi-Module Driving Assistant for Drowsiness

Enhancing Road Safety with AI: A Multi-Module Driving Assistant for Drowsiness

Muhammad Izzul Roslan, Norzihani Yusof*, Nuzulha Khilwani Ibrahim, Zuraini Othman

Fakulti Teknologi Maklumat dan Komunikasi, Universiti Teknikal Malaysia Melaka, 76100, Malacca, Malaysia

DOI: https://dx.doi.org/10.47772/IJRISS.2025.90900058

Received: 25 August 2025; Accepted: 02 September 2025; Published: 29 September 2025

ABSTRACT

One of the main causes of traffic accidents is sleepy driving, which has serious implications for drivers, passengers, and pedestrians. In this study an AI-powered multi-module driving assistant is presented with the goal of improving road safety through driver monitoring. The three main modules of the system are: AI Companion, which offers interactive engagement through natural language processing; Drowsiness Detection, which uses facial recognition techniques; and Music Recommendation, which uses machine learning for emotional state analysis. The Drowsiness Detection Module uses OpenCV and dlib’s facial landmark detection to track driver alertness in real time which implement alerts when fatigue symptoms are identified. While the Music Recommendation Module assesses emotional states to recommend suitable music for sustaining alertness and the AI Companion Module offers voice-based interaction and real-time assistance. In the system development Python, Flask, and a number of APIs that include OpenAI and OpenWeather. Result obtained in the testing phase revealed that dependable emotional state recognition, responsive conversational AI, and effective drowsiness detection accuracy. Future research will concentrate on increasing offline capabilities and enhancing performance in low light.

Keywords: Drowsy Driving, Machine Learning, Driver Assistant, Road Safety, Computer Vision, Facial Recognition

INTRODUCTION

Road safety remains a critical concern in Malaysia, where driver drowsiness significantly contributes to traffic accidents. According to the Royal Malaysia Police Traffic Investigation and Enforcement Department, approximately 20% of road accidents in Malaysia are attributed to drowsy driving annually [1].

The Malaysian Institute of Road Safety Research reports that between 2011 and 2021, fatigue contributed to 1,305 road fatalities nationwide [2]. Research indicates that prolonged driving under monotonous conditions significantly impairs driver alertness and performance, particularly affecting occupational drivers [3].

Drowsy driving impairs reaction times, decision-making abilities, and increases accident risks substantially. Studies demonstrate that fatigue-related crashes can be as dangerous as alcohol-impaired driving, with 20 hours of wakefulness producing performance deficits equivalent to a blood alcohol concentration of 0.08% [4]. Traditional preventive measures such as awareness campaigns and roadside warnings remain passive and fail to provide real-time driver monitoring [5].

The limitations of conventional approaches necessitate intelligent, proactive countermeasures. Artificial intelligence offers promising opportunities for real-time monitoring, engagement, and adaptive interventions to maintain driver alertness [6]. This study develops an AI-powered driving assistant with multi-functional modules specifically designed to combat driver drowsiness through integrated detection, engagement, and emotional support systems.

 Related Work

This section reviews relevant literature on drowsy driving risks, traditional fatigue management, AI-based drowsiness detection methods, and multimodal intelligent driving assistants.

Drowsy Driving and Accident Risk

Drowsy driving has been extensively documented as a critical risk factor in road accidents. Research consistently demonstrates that fatigue-related crashes can be as hazardous as alcohol-related incidents, with prolonged wakefulness producing performance deficits equivalent to legal intoxication [4]. Malaysian studies indicate that approximately 20% of road accidents are attributed to driver fatigue, particularly prevalent on highways during long-distance travel [1], [2]. Occupational drivers face heightened vulnerability due to extended periods of monotonous driving conditions [3].

Traditional Fatigue Management Approaches

Conventional drowsy driving prevention has relied primarily on passive measures including public awareness campaigns, roadside warning signs, and rest-stop advisories. While these strategies increase awareness, they do not actively monitor driver conditions in real-time [5]. Self-reporting methods and psychomotor vigilance tasks have been employed for fatigue evaluation, but these approaches are often subjective and prone to bias [7]. The gap between passive awareness and active intervention has prompted exploration of technology-driven solutions.

AI-Based Drowsiness Detection Methods

Recent advances in artificial intelligence and computer vision have enabled significant progress in real-time driver drowsiness detection. A study from previous has developed an AI-based sensor system that classified drowsy and non-drowsy states using machine learning algorithms, though with limited exploration of deep learning models [8]. Contemporary research has focused on lightweight frameworks optimized for real-time applications and edge device deployment [9].

Advanced approaches have incorporated multi-attention mechanisms and hybrid deep learning models combining Convolutional Neural Networks with temporal analysis for improved accuracy in complex environments [10]. Recent developments include VigilEye, an open-source real-time system integrating OpenCV with deep learning techniques for robust drowsiness detection under practical driving conditions [11]. These studies demonstrate AI’s potential in driver fatigue monitoring while highlighting ongoing challenges in low-light environments and resource-constrained systems.

Multimodal Driver Assistance Systems

Beyond standalone detection systems, research has explored multimodal intelligent driving assistants integrating drowsiness monitoring with complementary engagement strategies. Conversational AI has been applied to maintain driver attentiveness through voice-based interactions and contextual updates during extended journeys [12]. Affective computing techniques including emotion recognition have been employed to recommend personalized interventions such as music playlists to enhance driver mood and reduce fatigue [13].

These multimodal systems represent a shift from purely reactive detection toward proactive engagement and personalized support. However, existing work typically focuses on isolated functionalities rather than unified platforms combining detection, interaction, and mood regulation.

Summary of Review

The literature reflects substantial progress in the application of AI for drowsiness detection, with methods evolving from sensor-based monitoring to deep learning models capable of robust real-time performance. Traditional measures remain limited due to the not interactive method of drowsiness detection; while emerging multimodal approaches indicate the promise of integrating detection with driver engagement. Nevertheless, relatively few studies have attempted to combine multiple AI technologies into a holistic solution. This study addresses that gap by proposing an AI-powered multi-module driving assistant that integrates facial recognition-based drowsiness detection, conversational AI engagement, and emotion-based music recommendation into a unified system aimed at enhancing road safety.

METHODOLOGY

The AI-powered Multi-Module Driving Assistant was developed using a modular design approach to ensure scalability, adaptability, and real-time responsiveness. The methodology focuses on the overall system design, architecture, integrated modules, materials and tools used, workflow, and evaluation techniques. The proposed system integrates computer vision, machine learning, and natural language processing (NLP) technologies to address the problem of driver drowsiness and provide interactive engagement.

Overview of System Design

The system was designed as a modular framework that combines three core functionalities: drowsiness detection, AI-based conversational assistance, and emotion-driven music recommendation. Each module was implemented as an independent component, while communication and synchronization were managed through a Flask-based backend. This design allowed the system to operate efficiently in real time while providing flexibility for future enhancements and additional functionalities. The modular approach also ensured that errors in one component would not affect the performance of the entire system.

System Architecture

The architecture of the system follows a client–server model, where the frontend provides an interactive user interface while the backend manages data processing, AI model execution, and API communication. The frontend developed using HTML, CSS, and JavaScript, served as the primary medium for driver interaction. The backend, developed in Python with Flask, managed real-time facial recognition, speech processing, and music recommendation. RESTful API endpoints facilitated seamless communication between the two layers.

To enhance system intelligence, a number of external services were integrated into the architecture. The OpenAI API is used to generate conversational responses for the AI Companion module and the OpenWeather API delivered real-time weather updates. This integration ensured that the system provided not only safety alerts but also useful driving-related information. Figure 1 presents the diagrammatic representation of the system architecture.

Figure 1. System architecture workflow

System Module

The first core component of the system is the Drowsiness Detection Module, which monitors driver alertness through live video input. Facial recognition is performed using OpenCV and dlib’s 68-point facial landmark detection model. Key indicators such as the Eye Aspect Ratio (EAR) and Mouth Open Ratio (MOR) are calculated to identify early symptoms of fatigue, including prolonged blinking, yawning, and head tilting.

Prolonged blinking, yawning, or head tilting triggered both visual and audio alerts to ensure the driver’s immediate awareness. The presented formula 1.1 is how the EAR is calculated. An illustration of the EAR formula is shown as in visual explanation in Figure 2.

— (1.1)

Figure 2. Eye Aspect Ratio (EAR) visual explanation

The second module is the AI Companion, which functions as an intelligent conversational assistant. This module engages with drivers through voice-based communication powered by the OpenAI API. In addition to answering general queries, the assistant provides practical driving-related support such as weather updates retrieved via the OpenWeather API. This continuous interaction helps sustain driver attention during long or monotonous journeys.

The third module is the Music Recommendation Module analyzes the driver’s emotional state to suggest suitable playlists that help maintain alertness. Facial regions are first detected using OpenCV’s Haar Cascade, preprocessed into 48×48 grayscale images, and classified by a pre-trained CNN into categories such as happy or sad.

To ensure stability, predictions are filtered using a confidence threshold and temporal smoothing over multiple frames. The detected emotion is then mapped to mood-based song clusters created with KMeans using audio features such as valence, energy, and tempo, from which a playlist is generated. Recommended songs are displayed with direct YouTube links, enabling seamless playback and real-time adaptation to the driver’s mood.

Materials and Tools

The development of the system relies on a combination of open-source libraries, frameworks, and APIs that supported computer vision, natural language processing, and machine learning tasks. Python 3.11 served as the primary programming language, providing flexibility and wide compatibility with essential libraries. OpenCV and dlib were used for real-time video analysis and facial landmark detection, while Keras supported the training and execution of the CNN-based emotion classifier.

Data preprocessing and clustering were handled by scikit-learn, NumPy, and Pandas. For natural language processing and conversational responses, the OpenAI API was integrated alongside spaCy for lightweight text analysis. Flask was employed as the backend framework, with HTML, CSS, and JavaScript forming the frontend interface. External APIs such as OpenWeather provided real-time weather information, while the YouTube API was incorporated to enable seamless music playback.

The development and testing environment consisted of Visual Studio Code (VS Code) and Jupyter Notebook, which facilitated efficient debugging and prototyping. Together, these tools and resources ensured the system’s robustness, scalability, and adaptability across different platforms.

Table I. List of Materials and Tools

System Workflow

The system workflow is designed around a modular, user-driven interaction model, where each module operates independently and is activated based on the driver’s selection. At the start, the user selects the desired functionality from the interface which are Drowsiness Detection, AI Companion, or Music Recommendation. Once activated, the selected module takes full control of the input and processing pipeline until the session is terminated, or another module is chosen.

For example, when the Drowsiness Detection Module is selected, the webcam captures the driver’s facial features, and the system calculates the Eye Aspect Ratio (EAR) and Mouth Open Ratio (MOR) to monitor signs of fatigue, issuing visual and audio alerts when necessary. When the AI Companion Module is chosen, the system processes speech or text input using the OpenAI API, providing interactive conversational responses and real-time updates such as weather information.

Alternatively, if the Music Recommendation Module is activated, the driver’s emotion is detected using the CNN-based classifier, mapped into clusters with KMeans, and used to recommend appropriate songs with direct YouTube links for playback.

Analysis & Result

The AI-powered Multi-Module Driving Assistant was evaluated to assess its functionality, performance, and usability. The results are organized into three parts: (i) a summary of the modules and their key outputs, (ii) visual demonstration of the module interfaces, and (iii) detailed testing covering unit, integration, performance, usability, and system constraints.

System Module Overview

Table II provides an overview of the three core modules of the system, outlining their primary functionalities and the corresponding outputs observed during testing.

Table II. System Module Overview

Drowsiness Detection Module

The Drowsiness Detection Module is developed using OpenCV and dlib’s 68-point facial landmark detection model. It computes the Eye Aspect Ratio (EAR) and Mouth Open Ratio (MOR) to identify fatigue indicators such as prolonged blinking, yawning, or head tilting. When EAR values dropped below a predefined threshold, the system triggered alerts via on-screen messages and audible alarms.

Testing across various lighting conditions showed stable performance in well-lit environments, though accuracy decreased in low-light scenarios. The interface of the module, illustrated in Figure 3, demonstrates how fatigue warnings are displayed to the driver in real time.

Figure 3. Interface of the Drowsiness Detection Module

AI Companion Module

The AI Companion Module enabled drivers to interact through voice or text input. Powered by the OpenAI API, the module generated conversational responses while integrating external services such as OpenWeather for live weather updates. During testing, it successfully processed commands, answered general queries, and handled invalid or ambiguous inputs with error messages instead of system crashes. This enhanced driver engagement during long journeys. The interface of the AI Companion, shown in Figure 4, displays how driver inputs and system-generated responses appear on the user screen.

Figure 4. Interface of the AI Companion Module

Music Recommendation Module

The Music Recommendation Module maintained driver attentiveness by generating playlists based on detected emotions. Using a pre-trained CNN, the module classified facial expressions into emotional states such as happy or sad. These states were mapped into clusters using the KMeans algorithm to generate music recommendations aligned with the driver’s mood. Testing revealed that the emotion classifier is successful to give the playlist suggestions matched user expectations in most cases. The interface, presented in Figure 5, shows an example of the music playlist recommendations with embedded YouTube links for immediate playback.

Figure 5. Interface of the Music Recommendation Module

Testing Results

The system was evaluated through unit, integration, performance, and usability testing, each providing valuable insights into its functionality. Unit testing confirmed that all modules operated reliably when activated individually. The Drowsiness Detection Module accurately computed EAR and MOR values to identify drowsiness symptoms such as blinking and yawning, though its performance declined under dim lighting conditions. The AI Companion handled both valid and invalid inputs effectively, delivering meaningful responses without system crashes, while the Music Recommendation Module successfully classified emotions for smaller datasets but showed reduced efficiency with larger repositories.

Integration testing verified that modules could be selected and used sequentially without creating conflicts. Since the system follows a modular approach, users must choose one module at a time, for example enabling the Drowsiness Detection Module temporarily disables the AI Companion and Music Recommendation modules. This ensured that switching between features was smooth, with no overlap or interference between modules.

Performance testing demonstrated that each module responded in real time under standard conditions, although delays were observed when handling large music repositories or processing high API call volumes. Usability testing showed that the interface was intuitive and accessible across desktop and mobile platforms. However, limitations were noted: voice recognition accuracy decreased in noisy environments or with strong accents, and the Drowsiness Detection Module required adequate lighting and a good-quality camera. Despite these constraints, testers generally reported that the system was user-friendly, effective, and practical for improving driver alertness and engagement.

Summary of Findings

Overall, the evaluation demonstrated that the AI-powered Multi-Module Driving Assistant can effectively combine drowsiness detection, conversational interaction, and mood-based music recommendation into a cohesive system. The modules worked reliably for individually in a unified system, showing stable performance under most conditions. While the system’s reliance on external APIs and high-quality input devices posed certain limitations, the results confirm that it provides a non-intrusive, intelligent, and engaging solution for improving road safety. These findings highlight the potential of integrating AI-driven modules into driver assistance systems to reduce fatigue-related accidents and enhance the overall driving experience.

CONCLUSION

This study addresses the terrible consequences of sleepy driving, which killed 1,305 people between 2011 and 2021 and continues to cause one in every five road accidents countrywide. In this study we created an AI-powered driving assistant that actually cares about human lives including sleepiness detection, conversational help, and tailored music recommendations. In order to meet a goal that every number represents a person and traits driving tiredness as a very human problem including emotions, behavior, and psychological well-being. By utilizing standard cameras and open-source software, technology promotes equity in road safety measures across different socioeconomic levels, ensuring that life-saving interventions are not limited to expensive vehicles but can benefit all members of society.

The societal implication of this research is immediate accident prevention for Malaysia. The proposed work’s potential to reduce the of road accidents including emotional trauma for families that contributes quality of life in Malaysian communities. This study establishes important foundations for interdisciplinary research combining technology development with social science methodologies, opening avenues for future investigation into user acceptance patterns across demographic groups, cultural factors affecting technology adoption, and ethical implications of AI monitoring systems.

ACKNOWLEDGEMENT

The authors would like to thank the Centre of Research and Innovation Management of Universiti Teknikal Malaysia Melaka (UTeM) for sponsoring the publication fees under the Tabung Penerbitan CRIM UTeM.

REFERENCES

1. Royal Malaysia Police (PDRM) Bukit Aman Traffic Investigation and Enforcement Department, Road Accident Statistics Malaysia 2021. Kuala Lumpur: PDRM, 2022.
2. Malaysian Institute of Road Safety Research (MIROS), Annual Statistical Report on Road Accidents 2011–2021. Selangor: MIROS, 2022.
3. Kee, S. B. Md Tamrin, and Y. M. Goh, drivers in simulated prolonged driving,” Global Journal of Health Science, vol. 2, no. 1, pp. 167–176, Apr. 2010.
4. Dawson and K. Reid, “Fatigue, alcohol and performance impairment,” Nature, vol. 388, no. 6639, p. 235, Jul. 1997.
5. World Health Organization, Global Status Report on Road Safety 2018. Geneva: WHO, 2018.
6. A. El-Nabi, W. El-Shafai, E. S. M. El-Rabaie, K. F. Ramadan, F. E. Abd El-Samie, and S. Mohsen, “Machine learning and deep learning techniques for driver fatigue and drowsiness detection: A review,” Multimedia Tools and Applications, 2023. doi: 10.1007/s11042-023-15054-0.
7. Sleep Foundation, “Drowsy Driving vs Drunk Driving,” Nov. 3, 2023. [Online]. Available: https://www.sleepfoundation.org/drowsy-driving
8. Chinthalachervu, I. Teja, M. A. Kumar, N. S. Harshith, and T. S. Kumar, “Driver drowsiness detection and monitoring system using machine learning,” Journal of Physics: Conference Series, vol. 2325, no. 1, p. 012057, 2022. doi: 10.1088/1742-6596/2325/1/012057.
9. Albadawi, A. AlRedhaei, and M. Takruri, “Real-Time Machine Learning-Based Driver Drowsiness Detection Using Visual Features,” Journal of Imaging, vol. 9, no. 5, p. 91, 2023. doi: 10.3390/jimaging9050091.
10. Swathi et al., “Driver Drowsiness Monitoring System Using Visual Behavior And Machine Learning,” in 2022 5th International Conference on Multimedia, Signal Processing and Communication Technologies, 2022. doi: 10.1109/IMPACT55510.2022.10029275.
11. S. Sengar, A. Kumar, and O. Singh, “VigilEye — Artificial Intelligence-based Real-time Driver Drowsiness Detection,” arXiv preprint arXiv:2406.15646, 2024.
12. Upreti, “Comparative analysis of NLP algorithms for conversational AI assistants,” Master’s Thesis, Mälardalen University, Västerås, Sweden, 2023.
13. Jia, “A music emotion classification model based on the improved convolutional neural network,” Computational Intelligence and Neuroscience, vol. 2022, p. 6749622, 2022. doi: 10.1155/2022/6749622.