Legal Text Summarization using Bart and Explainable AI Techniques

Summarizing lengthy documents, especially in the legal domain, posed significant challenges for humans and automated systems. Human efforts entailed considerable time and effort, while automated systems sometimes faltered in decision-making, leading to ambiguity in the generated summaries. This research explored the use of text summarization in legal documentation coupled with an explainability feature. It addressed the challenges of condensing lengthy legal texts and improving automated summarization systems' transparency. The research involved gathering legal documents, developing a Bidirectional and Auto-Regressive Transformers (BART) summarization model, and integrating explainability within the system, visualizing the attention mechanism. The system performance, which included BERT Score, cosine similarity, and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score between human-generated and system-generated summaries, and evaluation by target users, led to several engaging insights on legal summarization. The model demonstrated moderate performance, where user feedback indicated satisfaction with its functionality but highlighted the need for user interface improvements. Future improvements were suggested, including refining model training, enhancing the user interface, and adding features like adjustable summary lengths and language translation.

Text summarization, Natural language processing, Explainable artificial intelligence, Legal Field, Bidirectional and auto-regressive transformers

1. V. Pandya, "Automatic Text Summarization of Legal Cases: A Hybrid Approach," in 5th International Conference on Advances in Computer Science and Information Technology, 2019, doi: 10.5121/csit.2019.91004. [Google Scholar] [Crossref]

2. D. Anand, and R. Wagh, "Effective deep learning approaches for summarization of legal texts," Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 5, pp. 2141–2150, May 2011, doi: 10.1016/j.jksuci.2019.11.015. [Google Scholar] [Crossref]

3. A. Shukla, P. Bhattacharya, S. Poddar, R. Mukherjee, K. Ghosh, P. Goyal, and S. Ghosh, "Legal Case Document Summarization: Extractive and Abstractive Methods and their Evaluation." in Proceedings of the conference of the Asia-Pacific chapter of the Association for Computational Linguistics and the international joint conference on natural language processing, Oct 2022, Available: https://arxiv.org/abs/2210.07544. [Google Scholar] [Crossref]

4. Adarsh, "Text Summarization with Attention based Networks" [Google Scholar] [Crossref]

5. https://medium.com/@iit2018056/text-summarizationwith-attention-based-networks-8492e9277c0 (accessed Apr, 29, 2024). [Google Scholar] [Crossref]

6. Z. Wang, "An Automatic Abstractive Text Summarization Model based on Hybrid Attention Mechanism," Journal of Physics: Conference Series, vol. 1848, no. 1, 012057, April 2021, doi: [Google Scholar] [Crossref]

7. 1088/1742-6596/1848/1/012057. [Google Scholar] [Crossref]

8. N, Alipour and S. Aydin, “Abstractive summarization using multilingual text-to-text transfer transformer for the Turkish text”, International Journal of Artificial Intelligence, vol. 14, no. 2, April 2025, pp. 1587-1596. [Google Scholar] [Crossref]

9. J. K. Adeniyi, S. A. Ajagbe, A. E. Adeniyi, H. O. Aworinde, P. B. Falola, and M. O. Adigun, “EASESUM: an online abstractive and extractive text summarizer using deep learning technique”, International Journal of Artificial Intelligence, vol. 13, no. 2, June 2024, pp. 1888-1899. [Google Scholar] [Crossref]

10. S. Dutta, A. K. Das, S. Ghosh, and D. Samanta, "Graph-based clustering technique for microblog clustering," in Data Analytics for Social Microblogging Platforms. Academic Press, 2023, pp. 165-192. Accessed: Apr, 29, 2024. [Online]. Available: https://books.google.com.my/. [Google Scholar] [Crossref]

11. N. A. Ranggianto, D. Purwitasari, C. Fatichah, R. W. Sholikah, Abstractive and Extractive Approaches for Summarizing Multi-document Travel Reviews. Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi). 2023 Dec 30;7(6), pp. 1464-75. [Google Scholar] [Crossref]

12. A. M. Zakariae, B. Frikh, and B. Ouhbi. "EXABSUM: a new text summarization approach for generating extractive and abstractive summaries." Journal of Big Data 10, no. 1, 2023, pp. 163. [Google Scholar] [Crossref]

13. N. Giarelis, C. Mastrokostas, and N. Karacapilidis, "Abstractive vs. Extractive Summarization: [Google Scholar] [Crossref]

14. An Experimental Review," Applied Sciences, vol. 13, no. 13, pp. 7620, 2023, doi: 10.3390/app13137620. [Google Scholar] [Crossref]

15. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, "BART: Denoising Sequence-to-Sequence Pretraining for Natural Language Generation, Translation, and Comprehension," in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 7871-7880 Available: https://arxiv.org/abs/1910.13461. [Google Scholar] [Crossref]

16. G. Erkan, and D. Radev, "LexRank: Graph-based Lexical Centrality as Salience in Text Summarization," Journal of Artificial Intelligence Research, vol. 22, pp. 457-479, 2004, doi: 10.1613/jair.1523. [Google Scholar] [Crossref]

17. D. R. Radev, and Z. Zhang, (2001, October 5). "Experiments in Single and MultiDocument Summarization Using MEAD. ResearchGate," in First document understanding conference, 2001, pp. 1-7. [Google Scholar] [Crossref]

18. K. Sharma, K. Singh, K. Sharma, and J. Gupta, "Question Summation and Sentence Similarity using BERT for Key Information Extraction." International Journal for Research in Applied Science and Engineering Technology, vol. 11, no. 4, pp. 1636–1639, 2023 doi: 10.22214/ijraset.2023.50087. [Google Scholar] [Crossref]

19. P. Watanangura, S. Vanichrudee, O. Minteer, T. Sringamdee, N. Thanngam, and T. Siriborvornratanakul. "A comparative survey of text summarization techniques." SN Computer Science, vol. 5, no. 1, 2023, pp 47. [Google Scholar] [Crossref]

20. R.D. Lins, H. Oliveira, and S.J. Simske, “Assessing the Reliability and Validity of the Measures for Automatic Text Summarization.” In Proceedings of the ACM Symposium on Document Engineering 2024, pp. 1-4. [Google Scholar] [Crossref]

21. B. Zhao, and Y.M. Lui, “Towards A Reliable Text Summarization Evaluation Metric Using Predictive Models”. International Journal of Pattern Recognition and Artificial Intelligence, vol. 36, no. 10, pp. 2251011, 2022. [Google Scholar] [Crossref]

22. A.M. Ibrahim, A. Marco and M. Aref. "A Systematic Review On Text Summarization Of Medical Research Articles." International Journal of Intelligent Computing and Information Sciences vol. 23, no. 2, pp. 50-61, 2023. [Google Scholar] [Crossref]

23. H. Özbolat, "Text Summarization: How to Calculate BertScore," [Google Scholar] [Crossref]

24. https://haticeozbolat17.medium.com/textsummarization-how-to-calculate-bertscore-771a51022964 (accessed Apr, 29, 2024). [Google Scholar] [Crossref]

25. A. Mulkar, "Explainable AI (xAI) in Natural Language Processing (NLP)," https://ankushmulkar.medium.com/explainable-ai-xai-innatural-language-processing-nlp-d75d5be216e3 (accessed Apr, 29, 2024). [Google Scholar] [Crossref]

26. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention Is All You Need," in Advances in Neural Information Processing Systems, vol. 30, 2017 Available: https://arxiv.org/abs/1706.03762 [Google Scholar] [Crossref]

27. N. H. A. M. Norkute, "Explainable AI for Text Summarization of Legal Documents," https://hyperight.com/explainable-ai-for-textsummarization-of-legal-documents/ (accessed Apr, 29, 2024). [Google Scholar] [Crossref]

28. "What are the pros and cons of using attention mechanisms in text summarization with RNNs?" www.linkedin.com. https://www.linkedin.com/advice/3/what-pros-cons-using-attentionmechanisms-textsummarization (accessed Apr, 28, 2024). [Google Scholar] [Crossref]

29. J. Vig, "Visualizing Attention in Transformer-Based Language Representation Models," 2019, Available: https://arxiv.org/abs/1904.02679. [Google Scholar] [Crossref]

30. D. Garreau, and U. Luxburg, "Explaining the Explainer: A First Theoretical Analysis of LIME," in Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics, 26–28 August 2020, vol 108, pp. 1287–1296 https://proceedings.mlr.press/v108/garreau20a.html [Google Scholar] [Crossref]

31. A. Shukla, P. Bhattacharya, S. Poddar, R. Mukherjee, K. Ghosh, P. Goyal, and S. Ghosh, "Legal Case Document Summarization: Extractive and Abstractive Methods and their Evaluation," in Proceedings of the conference of the Asia-Pacific chapter of the Association for Computational Linguistics and the international joint conference on natural language processing, 2022. doi: 10.5281/zenodo.7152317 [Google Scholar] [Crossref]

32. C. Burnell, J. Wood, M. Babin, S. Pesznecker, and N. Rosevear, "Writing Summaries," Pressbooks. https://openoregon.pressbooks.pub/wrd/chapter/writing-summaries/ (accessed May, 12, 2024). [Google Scholar] [Crossref]

33. M. H. Asif, and A. U. Yaseen, “Comparative Evaluation of Text Similarity Matrices for Enhanced Abstractive Summarization on CNN/Dailymail Corpus.” Journal of Computing & Biomedical Informatics, vol. 6, no. 01, 2023, pp. 208-215. [Google Scholar] [Crossref]

34. G. Sharma, and D. Sharma, “Automatic text summarization methods: A comprehensive review.” SN Computer Science, vol. 4, no. 1, 33, 2022. [Google Scholar] [Crossref]

35. T. Chellatamilan, S. K. Narayanasamy, L. G. K. Srinivasan, and S. MN. Islam. "Ensemble Text Summarization Model for COVID‐19‐Associated Datasets." International Journal of Intelligent Systems 2023, no. 1, 2023, pp. 3106631. [Google Scholar] [Crossref]

• The Role of Artificial Intelligence in Revolutionizing Library Services in Nairobi: Ethical Implications and Future Trends in User Interaction
• ESPYREAL: A Mobile Based Multi-Currency Identifier for Visually Impaired Individuals Using Convolutional Neural Network
• Comparative Analysis of AI-Driven IoT-Based Smart Agriculture Platforms with Blockchain-Enabled Marketplaces
• AI-Based Dish Recommender System for Reducing Fruit Waste through Spoilage Detection and Ripeness Assessment
• SEA-TALK: An AI-Powered Voice Translator and Southeast Asian Dialects Recognition