Inclusive Offline Multimodal Retrieval-Augmented Generation System for Accessible PDF-Based Knowledge Assistance

People with visual, speech and hearing impairment still face a major problem of receiving digital knowledge. In spite of the fact that the artificial intelligence enhances the information retrieval systems, the majority of the solutions are based on the cloud-based large language models and they do not offer an inclusive multimodal interaction. In this paper, an Offline Multimodal Retrieval-Augmented Generation (RAG) System is introduced that is intended to help differently-abled users to interact with PDF documents with the help of text, speech, and sign-language. The suggested system consists of the locally run large language model (LLaMA through Ollama), semantic retrieval based on FAISS, offline speech recognition, text-to-speech synthesis, and Sign animation rendering through gesture recognition. Our architecture is based on privacy, low latency and free deployment, unlike the traditional cloud-dependent AI assistants. Experimental evaluation demonstrates that the system effectively retrieves context-relevant responses while supporting voice interaction and visual gesture assistance. The proposed framework contributes toward inclusive AI-driven knowledge systems and demonstrates the feasibility of offline assistive intelligence platforms.

Accessibility AI, Retrieval-Augmented Generation, Offline LLM, Assistive Technology, Multimodal Interaction, Inclusive Computing

1. A. A. Khan, M. T. Hasan, K. Kemell, J. Rasku, and P. Abrahamsson, “Developing Retrieval-Augmented Generation (RAG) Based LLM Systems from PDFs: An Experience Report,” arXiv:2410.15944, 2024. [Google Scholar] [Crossref]

2. J. Swacha and M. Gracel, “Retrieval-Augmented Generation (RAG) Chatbots for Education: A Survey of Applications,” Applied Sciences, vol. 15, no. 8, p. 4234, 2025. [Google Scholar] [Crossref]

3. V. Vaidheeswaran and G. O. Nathan, “Need of Retrieval Augmented Generation for Conversational Assistants for Government Social Welfare Scheme Popularization,” International Journal of Scientific Research Archive, vol. 13, no. 1, pp. 1299–1312, 2024. [Google Scholar] [Crossref]

4. “Retrieval-Augmented Generation for Large Language Models in Healthcare: A Review,” PLOS Digital Health, 2025. [Google Scholar] [Crossref]

5. J. Li, Z. Li, and L. Sun, “Retrieval-Augmented Generation for Educational Applications,” Computers & Education: Artificial Intelligence, vol. 6, 100105, 2025. [Google Scholar] [Crossref]

6. S. Swacha and M. Gracel, “A Systematic Analysis of Retrieval-Augmented Generation Approaches for Medical Data in Resource-Constrained Environments,” Sensors, vol. 24, no. 6, 2024. [Google Scholar] [Crossref]

7. H. Sun, Y. Wang, and S. Zhang, “Retrieval-Augmented Generation for Domain-Specific Question Answering: A Case Study,” arXiv:2411.13691, 2024. [Google Scholar] [Crossref]

8. S. Kim, H. Jeon, D. Kim, M. Kim, and J. Kim, “HybridRAG: A Practical LLM-Based ChatBot Framework Based on Pre-Generated Q&A Over Raw Unstructured Documents,” arXiv, 2025. [Google Scholar] [Crossref]

9. Y. Zhu, “Large Language Models for Information Retrieval: A Survey,” ACM Computing Surveys, vol. 57, no. 5, 2025. [Google Scholar] [Crossref]

• A Machine Learning Model for Predicting the Risk of Developing Diabetes - T2DM Using Real-World Data from Kilifi, Kenya
• AI-Powered Facial Recognition Attendance System Using Deep Learning and Computer Vision
• A Comprehensive Review on Brain Tumour Segmentation Using Deep Learning Approach
• A Scalable Retrieval-Augmented Generation Pipeline for Domain-Specific Knowledge Applications
• Predictive Maintenance in Semiconductor Manufacturing Using Machine Learning on Imbalanced Dataset