A Hybrid EfficientNet and Self-Attention Architecture for Masked Face Recognition

The use of facial masks in the real-world setting has made the Masked Face Recognition (MFR) a critical research problem in Pattern Recognition. The classical face recognition technology is highly impaired in performance when the nose and mouth are covered as the main facial features. In this paper, a strong hybrid deep learning model will be suggested, which integrates the EfficientNet-B0 convolutional neural network with a self-attention mechanism to promote the learning of discriminative features on a partially visible part of the face. EfficientNet-B0 is also an efficient and scalable feature extractor, and the self-attention module allows global contextual reasoning and adaptable attention to unoccluded areas of the face, especially the periocular area. The suggested model is tested on the actual MFR2 dataset and has a recognition rate of 0.99, which proves to be better than the traditional CNN-based methods. The experimental test proves that the combination of self-attention can greatly enhance the resilience to the obstruction of the object by obstructing the features of the mask. The findings suggest that the hybrid architecture proposed is quite appropriate in real-time biometric authentication and surveillance and access control systems with masked environments.

Masked face recognition, Self-attention mechanism, EfficientNet-B0, Hybrid deep learning, MFR2 dataset.

1. W. Wan et al., "Masked face recognition via dual-branch convolutional self-attention network," Applied Soft Computing, 2024, doi: 10.1016/j.asoc.2024.112595. [Google Scholar] [Crossref]

2. A. F. et al., "Ensemble Learning using Transformers and Convolutional Networks for Masked Face Recognition," 2022, doi: 10.1109/sitis57111.2022.00070. [Google Scholar] [Crossref]

3. I. Hosen et al., "HiMFR: A hybrid masked face recognition through face inpainting," arXiv.org, 2022, doi: 10.48550/arXiv.2209.08930. [Google Scholar] [Crossref]

4. Mohammed R. Al-Sinan et al., "Ensemble Learning using Transformers and Convolutional Networks for Masked Face Recognition," in International Conference on Signal-Image Technology and Internet-Based Systems, 2022, doi: 10.1109/SITIS57111.2022.00070. [Google Scholar] [Crossref]

5. Z. Wang, G. Wang, B. Huang, Z. Xiong, Q. Hong, H. Wu, P. Yi, K. Jiang, N. Wang, Y. Pei, et al. Masked face recognition dataset and application. arXiv preprint arXiv:2003.09093, 2020. [Google Scholar] [Crossref]

6. Mohammed R. Al-Sinan et al., "Ensemble Learning using Transformers and Convolutional Networks for Masked Face Recognition," 2022, doi: 10.48550/arxiv.2210.04816. [Google Scholar] [Crossref]

7. M. Rodrigo et al., "Comprehensive comparison between vision transformers and convolutional neural networks for face recognition tasks," Dental Science Reports, 2024, doi: 10.1038/s41598-024-72254-w. [Google Scholar] [Crossref]

8. I. Hosen et al., "HiMFR: A Hybrid Masked Face Recognition Through Face Inpainting," 2022. [Google Scholar] [Crossref]

9. Y. Ge et al., "Masked face recognition with convolutional visual self-attention network," Neurocomputing, 2022, doi: 10.1016/j.neucom.2022.10.025. [Google Scholar] [Crossref]

10. W. Zhao et al., "Masked Face Transformer," IEEE Transactions on Information Forensics and Security, doi: 10.1109/tifs.2023.3322600. [Google Scholar] [Crossref]

11. Y. Zhu et al., "Joint holistic and masked face recognition," IEEE Transactions on Information Forensics and Security, doi: 10.1109/TIFS.2023.3280717. [Google Scholar] [Crossref]

12. W. Wan et al., "Masked face recognition based on knowledge distillation and convolutional self-attention network," International Journal of Machine Learning and Cybernetics, 2024, doi: 10.1007/s13042-024-02390-2. [Google Scholar] [Crossref]

13. L. Ouannes et al., "Enhancing Face Recognition in Degraded Conditions via Vision Transformer," 2024, doi: 10.1109/codit62066.2024.10708244. [Google Scholar] [Crossref]

14. L. C. Tiong et al., "Flexible Biometrics Recognition: Bridging the Multimodality Gap Through Attention, Alignment and Prompt Tuning," 2024, doi: 10.1109/cvpr52733.2024.00033. [Google Scholar] [Crossref]

15. Y. Wang et al., "Learning 3d face representation with vision transformer for masked face recognition," 2022, doi: 10.1109/CACML55074.2022.00092. [Google Scholar] [Crossref]

16. K. Wang et al., "FaceMAE: Privacy-Preserving Face Recognition via Masked Autoencoders," 2022. [Google Scholar] [Crossref]

17. J. Guo et al., "Face Recognition System with Occlusion Based on Attention Mechanism Improvement of the Vision Mamba Model," 2025, doi: 10.1109/icaisisas64483.2025.11051738. [Google Scholar] [Crossref]

18. H. V. Phan et al., "Fast and interpretable face identification for out-of-distribution data using vision transformers," arXiv.org, 2023, doi: 10.48550/arxiv.2311.02803. [Google Scholar] [Crossref]

19. A. Zhu et al., "Cross-Task Multi-Branch Vision Transformer for Facial Expression and Mask Wearing Classification," 2024, doi: 10.48550/arxiv.2404.14606. [Google Scholar] [Crossref]

20. W. Chang et al., "ResSaNet: A Hybrid Backbone of Residual Block and Self-Attention Module for Masked Face Recognition," in International Conference on Computer Vision, 2021, doi: 10.1109/ICCVW54120.2021.00170. [Google Scholar] [Crossref]

21. S. Lee et al., "Latent-OFER: Detect, Mask, and Reconstruct with Latent Vectors for Occluded Facial Expression Recognition," 2023, doi: 10.1109/iccv51070.2023.00148. [Google Scholar] [Crossref]

22. V. Vidal et al., "A benchmark on masked face recognition," in SIBGRAPI Conference on Graphics, Patterns and Images, 2022, doi: 10.1109/SIBGRAPI55357.2022.9991785. [Google Scholar] [Crossref]

23. Y. Xu, "Based on the contrastive learning classifier for occluded face recognition," Procedia Computer Science, 2025, doi: 10.1016/j.procs.2025.08.148. [Google Scholar] [Crossref]

24. Y. Liu, "Masked face recognition based on transfer learning." [Google Scholar] [Crossref]

25. A. Iftikhar et al., "Masked Face Detection and Recognition Using a Unified Feature Extractor," 2024, doi: 10.1109/icacs60934.2024.10473243. [Google Scholar] [Crossref]

26. S. Yang et al., "Hybrid Architecture-Based Evolutionary Robust Neural Architecture Search," IEEE Transactions on Emerging Topics in Computational Intelligence, 2024, doi: 10.1109/tetci.2024.3400867. [Google Scholar] [Crossref]

27. Q. Wang et al., "AAN-Face: Attention Augmented Networks for Face Recognition," IEEE Transactions on Image Processing, 2021, doi: 10.1109/TIP.2021.3107238. [Google Scholar] [Crossref]

28. H. Qiu et al., "End2End Occluded Face Recognition by Masking Corrupted Features," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, doi: 10.1109/TPAMI.2021.3098962. [Google Scholar] [Crossref]

29. Q. Wang et al., "DSA-Face: Diverse and Sparse Attentions for Face Recognition Robust to Pose Variation and Occlusion," IEEE Transactions on Information Forensics and Security, 2021, doi: 10.1109/TIFS.2021.3109463. [Google Scholar] [Crossref]

30. A. George et al., "EdgeFace: Efficient Face Recognition Model for Edge Devices," 2023. [Google Scholar] [Crossref]

• What the Desert Fathers Teach Data Scientists: Ancient Ascetic Principles for Ethical Machine-Learning Practice
• Comparative Analysis of Some Machine Learning Algorithms for the Classification of Ransomware
• Comparative Performance Analysis of Some Priority Queue Variants in Dijkstra’s Algorithm
• Transfer Learning in Detecting E-Assessment Malpractice from a Proctored Video Recordings.
• Dual-Modal Detection of Parkinson’s Disease: A Clinical Framework and Deep Learning Approach Using NeuroParkNet