Optimized Multimodal Human Activity and Anomaly Detection

Human activity recognition and anomaly detection play a crucial role in applications such as intelligent surveillance, healthcare monitoring, and smart environments. This study proposes a optimized multimodal framework that integrates video data and IMU sensor signals to differentiate normal and abnormal human activities efficiently. Each data stream is processed independently using optimized, low-complexity deep learning models, and predictions are combined at the decision level to enhance accuracy while avoiding the overhead of feature fusion. Model Optimization reduces memory usage, model size, and inference time, enabling deployment on edge devices such as smart cameras, smartwatches, and smartphones. The system is evaluated on various activities, including fall, driver drowsiness as well as normal activity like walk, run etc, under diverse lighting conditions, sensor placements, and environmental variations to ensure robust performance. The framework emphasizes real-time monitoring and low-latency response, providing a scalable and practical solution for continuous anomaly detection. Future work may extend the system to additional modalities and incremental learning for improved adaptability.

Anomaly Detection, Multimodal Learning, Op- timization

1. Shyma Zaidi, Jagadeesh B, Sudheesh K V, Audre Arlene A. ”Video Anomaly Detection and Human Activity Classification for Recognition.” IEEE Conference on Computational Intelligence and Communication Networks (CICN), 2017. [Google Scholar] [Crossref]

2. Zhenyu Shi, J. Andrew Zhang, R. Xu, and G. Fang. ”Human Activity Recognition Using Deep Learning Networks with Enhanced Channel State Information.” IEEE Transactions on Cognitive Communications and Networking, vol. 4, no. 2, pp. 257–265, 2018. [Google Scholar] [Crossref]

3. Aikaterini Tsanousa, Symeon Vrochidis, Georgios Meditskos, Ioannis Kompatsiaris. ”A Weighted Late Fusion Framework for Recognizing Human Activity from Wearable Sensors.” IEEE Access, vol. 7, pp. 80673–80684, 2019. [Google Scholar] [Crossref]

4. D. Miki, S. Chen, and K. Demachi. ”Unnatural Human Motion De- tection Using Weakly Supervised DNN.” IEEE Access, vol. 8, pp. 194998–195010, 2020. [Google Scholar] [Crossref]

5. J. Yan, F. Angelini, and S. M. Naqvi. ”Privacy-Preserving Human Action Recognition for Anomaly Detection.” IEEE Transactions on Image Processing, vol. 29, pp. 8174–8188, 2020. [Google Scholar] [Crossref]

6. H. Lv, P. Yi, R. Liu, Y. Hou, D. Zhou, Q. Zhang, and X. Wei. ”Asymmetric Anomaly Detection for Human-Robot Interaction.” IEEE Transactions on Industrial Informatics, vol. 17, no. 9, pp. 6152–6162, 2021. [Google Scholar] [Crossref]

7. H. E. Azzag, I. E. Zeroual, and A. Ladjailia. ”Real-Time Human Action Recognition Using Deep Learning.” IEEE Access, vol. 10, pp. 22983–22992, 2022. [Google Scholar] [Crossref]

8. L. Yashaswi, S. Mekala, and M. D. Prasad. ”Human Anomaly Detection Using Deep Learning.” IEEE Xplore, 2023. [Google Scholar] [Crossref]

9. H.-J. Jeon, S. Lang, C. Vogel, and R. Behrens. ”Anomaly Detection from Image Classification.” IEEE Access, vol. 12, pp. 4571–4584, 2024.S. A. Roseline, S. Karthik, and I. N. V. D. Sruti. ”Intelligent Human Anomaly Detection Using LSTM Autoencoders.” IEEE Conference on Emerging Trends in Artificial Intelligence and Data Science (ETAIDS), 2024. [Google Scholar] [Crossref]

• The Role of Artificial Intelligence in Revolutionizing Library Services in Nairobi: Ethical Implications and Future Trends in User Interaction
• ESPYREAL: A Mobile Based Multi-Currency Identifier for Visually Impaired Individuals Using Convolutional Neural Network
• Comparative Analysis of AI-Driven IoT-Based Smart Agriculture Platforms with Blockchain-Enabled Marketplaces
• AI-Based Dish Recommender System for Reducing Fruit Waste through Spoilage Detection and Ripeness Assessment
• SEA-TALK: An AI-Powered Voice Translator and Southeast Asian Dialects Recognition