Enhanced Retrieval-Augmented Generation Framework for Intelligent Multi-Document Question Answering

Retrieval-Augmented Generation (RAG) improves Large Language Models (LLMs) by using external documents to support their answers. However, baseline RAG architectures are limited by single-modality retrieval, fixed-size chunking, and lack of hallucination monitoring. This paper introduces an advanced hybrid RAG framework for multi-document question answering, enhancing retrieval quality, contextual coherence, and response fidelity.The proposed system combines FAISS’s dense semantic retrieval with BAAI/bge-large-en-v1.5 embeddings and BM25Okapi’s sparse lexical retrieval. Reciprocal Rank Fusion (RRF) combines results from both modalities to improve recall without changing any parameters. A semantic chunking strategy is introduced to keep the meaning of documents. This strategy uses sentence-level embeddings and percentile-based breakpoint detection to adaptively split documents. A cross-encoder reranker (ms-marco-MiniLM-L-12-v2) is used to improve the relevance scoring of the retrieved candidates.To mitigate hallucination without additional computational overhead, a reference-free faithfulness score is calculated by comparing the cosine similarity of generated responses to retrieved context embeddings. A multiprovider LLM abstraction layer makes sure that different cloud models are all based on the same things. The system is evaluated using Recall@K, Mean Reciprocal Rank (MRR), Precision@K, faithfulness score, and end-to-end latency. This shows that it is better at retrieving information and generating grounded information than dense-only baselines.

Retrieval-Augmented Generation, Hybrid Retrieval, FAISS, BM25, Semantic Chunking, Cross-Encoder Reranking, Hallucination Mitigation, Multi-Document Question Answering.

1. R. Hu, S. Liu, P. Qi, J. Liu and F. Li, ”ICCA-RAG: Intelligent Customs Clearance Assistant Using Retrieval-Augmented Generation (RAG),” in IEEE Access, vol. 13, pp. 39711-39726, 2025, doi: 10.1109/ACCESS.2025.3544408. [Google Scholar] [Crossref]

2. B. Saha, U. Saha and M. Zubair Malik, ”QuIM-RAG: Advancing Retrieval-Augmented Generation With Inverted Question Matching for Enhanced QA Performance,” in IEEE Access, vol. 12, pp. 185401185410, 2024, doi: 10.1109/ACCESS.2024.3513155. [Google Scholar] [Crossref]

3. M. Hindi, L. Mohammed, O. Maaz and A. Alwarafy, ”Enhancing the Precision and Interpretability of Retrieval-Augmented Generation (RAG) in Legal Technology: A Survey,” in IEEE Access, vol. 13, pp. 4617146189, 2025, doi: 10.1109/ACCESS.2025.3550145. [Google Scholar] [Crossref]

4. M. V. Jadeja, M. J. Lad and P. Kamboj, ”Optimizing Response Precision:A Dual-Model Chatbot Using RAG and NLP,” 2025 World Skills Conference on Universal Data Analytics and Sciences (WorldSUAS), Indore, India, 2025, pp. 1-5, doi: 10.1109/WorldSUAS66815.2025. [Google Scholar] [Crossref]

5. V. K. Singh, Gaurang and S. Kumar, ”A PDF Chatbot System: Enhancing Document Interaction with Natural Language Processing,” 2025 4th OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 5.0, Raigarh, India, 2025, pp. 1-5, doi: 10.1109/OTCON65728.2025.11070367. [Google Scholar] [Crossref]

6. S. Siddique and F. Alsayoud, ”Multi-Tiered RAG-Based Chatbot for [Google Scholar] [Crossref]

7. Mental Health Support,” 2025 Eighth International Women in Data Science Conference at Prince Sultan University (WiDS PSU), Riyadh, Saudi Arabia, 2025, pp. 1-6, doi: 10.1109/WiDS-PSU64963.2025.00041. [Google Scholar] [Crossref]

8. H. D. Hua, C. P. Yin, W. P. Voon and H. X. Chun, ”RAG-Enhanced AI Agent Chatbot Architecture for Optimizing Insurance Operations,” 2025 IEEE 7th Symposium on Computers and Informatics (ISCI), Kuala Lumpur, Malaysia, 2025, pp. 15-20, doi: 10.1109/ISCI65687.2025.11167672. [Google Scholar] [Crossref]

9. D. Shikha, S. Bansal, A. Raman and S. Sharma, ”QuickAid: A Hybrid RAG and LLM Framework for Improving Chatbot Accuracy and Relevance in First-Aid Guidance,” 2025 International Conference onComputing and Communication Technologies (ICCCT), Chennai, India, 2025, pp. 1-5, doi: 10.1109/ICCCT63501.2025.11019682. [Google Scholar] [Crossref]

10. M. A. Khadija, A. Aziz and W. Nurharjadmo, ”Automating Information Retrieval from Faculty Guidelines: Designing a PDF-Driven Chatbot powered by OpenAI ChatGPT,” 2023 International Conference on Computer, Control, Informatics and its Applications (IC3INA), Bandung, Indonesia, 2023, pp. 394-399, doi: 10.1109/IC3INA60834.2023.10285808. [Google Scholar] [Crossref]

11. T. Prem Jacob, B. L. S. Bizotto and M. Sathiyanarayanan, ”Constructing the ChatGPT for PDF Files with Langchain – AI,” 2024 International Conference on Inventive Computation Technologies (ICICT), Lalitpur, Nepal, 2024, pp. 835-839, doi: 10.1109/ICICT60155.2024.10544643. [Google Scholar] [Crossref]

12. C. J. Silaen, T. Mantoro, M. A. Ayu and R. T. Handayanto, ”Automatic Generation of Presentation Slides from PDF Using Retrieval-Augmented Chatbot,” 2025 IEEE 11th International Conference on Computing, Engineering and Design (ICCED), Cairo, Egypt, 2025, pp. 1-6, doi: 10.1109/ICCED68324.2025.11324852. [Google Scholar] [Crossref]

13. M.P.Geetha, G. Thirukumaran, C.Pradakshana, B.Sudharsana and T. Ashwin, ”Conversational AI Meets Documents Revolutionizing PDFInteraction with GenAI,” 2024 International Conference on Emerging Research in Computational Science (ICERCS), Coimbatore, India, 2024, pp. 1-6, doi: 10.1109/ICERCS63125.2024.10895303. [Google Scholar] [Crossref]

14. S. Uke, A. Laddha, S. Bambal, A. Bonde and P. Atram, ”Generative AIPowered Conversational Assistant for Document, Web, and RepositoryInteraction and Retrieval,” 2025 International Conference on Computing Technologies and Data Communication (ICCTDC), HASSAN, India, 2025, pp. 1-5, doi: 10.1109/ICCTDC64446.2025.11158845. [Google Scholar] [Crossref]

15. K. Anvitha, T. Durjay, K. Sathvika, G. Gnanendra, A. S and S. K. Natarajan, ”EduBot: A Compact AI-Driven Study Assistant for Contextual Knowledge Retrieval,” 2025 Global Conference in Emerging Technology (GINOTECH), PUNE, India, 2025, pp. 1-7, doi: 10.1109/GINOTECH63460.2025.11077097. [Google Scholar] [Crossref]

16. R. More, ”A Unified Evaluation Framework for Grounded LLM Architectures: Comparative Analysis of RAG, Self-RAG, and Agentic RAG,” 2025 5th International Conference on Artificial Intelligenceand Signal Processing (AISP), Vijayawada, India, 2025, pp. 1-5, doi: 10.1109/AISP68263.2025.11396270. [Google Scholar] [Crossref]

17. S. Vakayil, D. S. Juliet, A. J and S. Vakayil, ”RAG-Based LLM Chatbot Using Llama-2,” 2024 7th International Conference on Devices, Circuits and Systems (ICDCS), Coimbatore, India, 2024, pp. 1-5, doi: 10.1109/ICDCS59278.2024.10561020. [Google Scholar] [Crossref]

18. D. Shikha, S. Bansal, A. Raman and S. Sharma, ”QuickAid: A Hybrid RAG and LLM Framework for Improving Chatbot Accuracy and Relevance in First-Aid Guidance,” 2025 International Conference onComputing and Communication Technologies (ICCCT), Chennai, India, 2025, pp. 1-5, doi: 10.1109/ICCCT63501.2025.11019682. [Google Scholar] [Crossref]

19. H. -C. Lee, K. Hung, G. M. -T. Man, R. Ho and M. Leung, ”Development of an RAG-Based LLM Chatbot for Enhancing Technical Support Service,” TENCON 2024 - 2024 IEEE Region 10 Conference (TENCON), Singapore, Singapore, 2024, pp. 1080-1083, doi: 10.1109/TENCON61640.2024.10902801. [Google Scholar] [Crossref]

20. A. Ilapaka and R. Ghosh, ”A Comprehensive RAG-Based LLM for AI-Driven Mental Health Chatbot,” 2025 7th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA), Ankara, Turkiye, 2025, pp. 1-5, doi: 10.1109/ICHORA65333.2025.11017017. [Google Scholar] [Crossref]

21. A. Sar et al., ”PDF-Based Chatbot Development Using LLAMA2 and LangChain: Training and Deployment for Document Interaction,” 2024 OPJU International Technology Conference (OTCON) on Smart Computing for Innovation and Advancement in Industry 4.0, Raigarh, India, 2024, pp. 1-6, doi: 10.1109/OTCON60325.2024.10688337. [Google Scholar] [Crossref]

22. M. R. Putri, A. Y. Husodo and B. Irmawati, ”Simplification of Embedding Process in Retrieval Augmented Generation for Optimizing Question Answering Chatbot Model,” 2024 IEEE International Conference on Communication, Networks and Satellite (COMNETSAT), Mataram, Indonesia, 2024, pp. 665-670, doi: 10.1109/COMNETSAT63286.2024.10862926. [Google Scholar] [Crossref]

23. M. I. S. Watson Benjamin, A. J. V. K, R. Yadav, V. S. ReddyThippareddy and B. Raju, ”Context-Fusion: An Intelligent Retrieval-Augmented Conversational AI Framework with Multi-Model Support,” 2025 3rd International Conference on Self Sustainable Artificial Intelligence Systems (ICSSAS), Erode, India, 2025, pp. 965-971, doi: 10.1109/ICSSAS66150.2025.11081255. [Google Scholar] [Crossref]

24. O. T. -C. Chen and C. -H. Chang, ”Enhancing Medical QuestionAnswering Chatbot with RAG-Based Retrieval and Summarization,” 2025 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan), Kaohsiung, Taiwan, 2025, pp. 209-210, doi: 10.1109/ICCE-Taiwan66881.2025.11207819. [Google Scholar] [Crossref]

25. O. T. -C. Chen and C. -H. Chang, ”Enhancing Medical QuestionAnswering Chatbot with RAG-Based Retrieval and Summarization,” 2025 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Taiwan), Kaohsiung, Taiwan, 2025, pp. 209-210, doi: 10.1109/ICCE-Taiwan66881.2025.11207819. [Google Scholar] [Crossref]

26. H. -C. Lee, K. Hung, G. M. -T. Man, R. Ho and M. Leung, ”Development of an RAG-Based LLM Chatbot for Enhancing Technical Support Service,” TENCON 2024 - 2024 IEEE Region 10 Conference (TENCON), Singapore, Singapore, 2024, pp. 1080-1083, doi:10.1109/TENCON61640.2024.10902801. [Google Scholar] [Crossref]

• Assessment of the Role of Artificial Intelligence in Repositioning TVET for Economic Development in Nigeria
• Teachers’ Use of Assure Model Instructional Design on Learners’ Problem Solving Efficacy in Secondary Schools in Bungoma County, Kenya
• “E-Booksan Ang Kaalaman”: Development, Validation, and Utilization of Electronic Book in Academic Performance of Grade 9 Students in Social Studies
• Analyzing EFL University Students’ Academic Speaking Skills Through Self-Recorded Video Presentation
• Major Findings of The Study on Total Quality Management in Teachers’ Education Institutions (TEIs) In Assam – An Evaluative Study