Blockchain-Based Multimodal Misinformation Detection System

Digital misinformation rapidly spreading exponentially through social media platforms has generated the dire need to have a strong, correct, and transparent detection system in place. The current methods are mostly based on single-model machine learning methods on text-based information, which is not effective in detecting multimodal falsehoods like deceptive memes that integrate images with text. Moreover, centralized storage systems make such systems vulnerable to manipulation of data and they are not very transparent. The given paper features a proposal of a Blockchain-Based Multimodal Misinformation Detection System, which incorporates the use of Convolutional Neural Network (CNN)-based image feature extraction, Optical Character Recognition (OCR), transformer-based text encoding, and feature fusion to classify memes. A purpose-built Fact-Checking Module uses Large Language Model (LLM) reasoning in conjunction with real-time news retrieval to contextually verify textual assertions. An explainability layer calculates the sentiment, toxicity, readability, and perplexity scores to increase the interpretability. Final prediction results are then transformed into SHA-256 cryptographic hashes and stored forever in an Ethereum blockchain as smart contracts, which guarantee auditability immutability. It follows the MERN stack upon Flask-based AI microservice to implement the system. The validation accuracy, the F1-score, and the precision of the traditional unimodal baselines are lower than those of the experimental evaluation (93.9% and 0.935, respectively). The suggested framework will bring the state-of-the-art of multimodal detection, contextual fact verification, explainable AI, and decentralized verification into a single and scalable solution.

Misinformation Detection, Blockchain, Multimodal Learning, Fact-Checking, Large Language Model, Explainable AI.

1. V. Rubin, N. Conroy, Y. Chen, and S. Cornwell, "Fake news or truth? Using satirical cues to detect potentially misleading news," in Proc. NAACL Workshop Comput. Approaches Deception Detection, 2016, pp. 7–17. [Google Scholar] [Crossref]

2. R. Zafarani, M. A. Abbasi, and H. Liu, Social Media Mining: An Introduction. Cambridge University Press, 2014. [Google Scholar] [Crossref]

3. M. Granik and V. Mesyura, "Fake news detection using naive Bayes classifier," in Proc. IEEE 1st Ukraine Conf. Elect. Comput. Eng., 2017, pp. 900–903. [Google Scholar] [Crossref]

4. S. Nakamoto, "Bitcoin: A peer-to-peer electronic cash system," Decentralized Business Review, 2008. [Google Scholar] [Crossref]

5. Y. Qi, J. Cao, and T. Sheng, "Exploiting multi-domain visual information for fake news detection," in Proc. IEEE ICDM, 2019, pp. 518–527. [Google Scholar] [Crossref]

6. Shu, A. Sliva, S. Wang, J. Tang, and H. Liu, "Fake news detection on social media: A data mining perspective," ACM SIGKDD Explorations Newsletter, vol. 19, no. 1, pp. 22–36, 2017. [Google Scholar] [Crossref]

7. H. Ahmed, I. Traore, and S. Saad, "Detection of online fake news using n-gram analysis and machine learning techniques," in Proc. Int. Conf. Intelligent, Secure, Dependable Systems, 2017, pp. 127–138. [Google Scholar] [Crossref]

8. Y. Kim, "Convolutional neural networks for sentence classification," in Proc. EMNLP, 2014, pp. 1746–1751. [Google Scholar] [Crossref]

9. J. Devlin, M. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," in Proc. NAACL-HLT, 2019, pp. 4171–4186. [Google Scholar] [Crossref]

10. N. Sharma, R. Rai, and P. Agarwal, "Automatic meme generation and classification using deep learning," Int. J. Advanced Comput. Sci. Appl., vol. 11, no. 7, pp. 354–361, 2020. [Google Scholar] [Crossref]

11. D. Kiela, H. Firooz, A. Mohan, V. Raykar, A. Saha, P. Grave, and M. Bethge, "The Hateful Memes Challenge: Detecting hate speech in multimodal memes," in Proc. NeurIPS, 2020, pp. 2611–2624. [Google Scholar] [Crossref]

12. T. Brown et al., "Language models are few-shot learners," in Proc. NeurIPS, vol. 33, 2020, pp. 1877–1901. [Google Scholar] [Crossref]

13. J. Thorne, A. Vlachos, C. Christodoulopoulos, and A. Mittal, "FEVER: A large-scale dataset for fact extraction and verification," in Proc. NAACL-HLT, 2018, pp. 809–819. [Google Scholar] [Crossref]

14. V. Buterin, "A next-generation smart contract and decentralized application platform," Ethereum White Paper, 2014. [Google Scholar] [Crossref]

15. G. Li, G. Zhou, and C. Li, "Blockchain for misinformation detection: Architecture and implementation," IEEE Access, vol. 9, pp. 112341–112352, 2021. [Google Scholar] [Crossref]

• A Machine Learning Model for Predicting the Risk of Developing Diabetes - T2DM Using Real-World Data from Kilifi, Kenya
• AI-Powered Facial Recognition Attendance System Using Deep Learning and Computer Vision
• A Comprehensive Review on Brain Tumour Segmentation Using Deep Learning Approach
• A Scalable Retrieval-Augmented Generation Pipeline for Domain-Specific Knowledge Applications
• Predictive Maintenance in Semiconductor Manufacturing Using Machine Learning on Imbalanced Dataset