Revolutionizing Skin Cancer Diagnosis with AI: Integrating YOLO-Based Lesion Detection into a Telegram Chatbot

Skin cancer is one of the leading types of cancer globally and the early detection and treatment of skin lesions can drastically decrease mortality. Traditional diagnostic methods require specialized dermatological knowledge and facilities, which are inaccessible in remote and underserved areas. In this paper we develop an Artificial Intelligence-based skin cancer screening framework which is comprised of a YOLO-based object detection model integrated with a Telegram bot for real-time lesion analysis. Using the ISIC (International Skin Imaging Collaboration) dataset, we performed data labelling using Roboflow and developed a YOLO-based object detection model trained to detect nine classes of skin lesions namely: melanoma, basal cell carcinoma, squamous cell carcinoma, actinic keratosis, among others clinically relevant types of lesions. We then integrated our trained model into a Telegram bot which allows for the real-time assessment of uploaded lesion images. Through rigorous testing on the validation set, we report an mAP@50 of 66.4% and an accuracy of recall of 81.0%, precision of 52.5% and an F1 score of 63.7%. This work suggests a feasible approach in using Computer Vision and conversational AI for remote preliminary dermatological screening.

Skin Cancer Detection, YOLO, Computer Vision, Deep Learning, Telegram Chatbot, Teledermatology, Roboflow, Artificial Intelligence.

1. G. Jocher, A. Chaurasia, and J. Qiu, “YOLO by Ultralytics,” GitHub Repository, 2023. Available: https://github.com/ultralytics/ultralytics [Google Scholar] [Crossref]

2. IEEE Access Editorial Board, “Recent Advances in Artificial Intelligence for Medical Image Analysis,” IEEE Access, vol. 10, pp. 45000–45025, 2022. [Google Scholar] [Crossref]

3. Sarvamangala, D. and Kulkarni, R. V. (2022). Convolutional neural networks in medical image understanding: a survey. Evolutionary intelligence, 15(1):1–22. [Google Scholar] [Crossref]

4. Rothman, D. and Gulli, A. (2022). Transformers for Natural Language Processing: Build, train, and fine-tune deep neural network architectures for NLP with Python, PyTorch, TensorFlow, BERT, and GPT-3. Packt Publishing Ltd. [Google Scholar] [Crossref]

5. Gaafar, A. S., Dahr, J. M., and Hamoud, A. K. (2022). Comparative analysis of performance of deep learning classification approach based on lstm-rnn for textual and image datasets. Informatica, [Google Scholar] [Crossref]

6. A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale,” International Conference on Learning Representations (ICLR), 2021. [Google Scholar] [Crossref]

7. M. Kavitha, P. V. V. S. Srinivas, P. S. L. Kalyampudi, C. S. F and S. Srinivasulu, "Machine Learning Techniques for Anomaly Detection in Smart Healthcare," 2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2021, pp. 1350-1356,doi:0.1109/ICIRCA51532.2021.9544795 [Google Scholar] [Crossref]

8. Jiang, P., Ergu, D., Liu, F., Cai, Y., and Ma, B. (2022). Areview of yolo algorithm developments. Procedia Computer Science, 199:1066–1073. [Google Scholar] [Crossref]

9. Rao, TV Madhusudhana, P. Srinivasa Rao, and PS Latha Kalyampudi. "Iridology based vital organs malfunctioning identification using machine learning techniques." International Journal of Advanced Science and Technology 29.5 (2020): 5544-5554. [Google Scholar] [Crossref]

10. M. Tan and Q. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” Proceedings of the International Conference on Machine Learning (ICML), pp. 6105–6114, 2019 [Google Scholar] [Crossref]

11. N. C. F. Codella, D. Gutman, M. Celebi, et al., “Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging,” ISBI, pp. 168–172, 2018. [Google Scholar] [Crossref]

12. P. Tschandl, C. Rosendahl, and H. Kittler, “The HAM10000 Dataset: A Large Collection of Multi-Source Dermatoscopic Images of Common Pigmented Skin Lesions,” Scientific Data, vol. 5, no. 180161, 2018. [Google Scholar] [Crossref]

13. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779–788, 2016. [Google Scholar] [Crossref]

14. A. Esteva, B. Kuprel, R. A. Novoa, et al., “Dermatologist-Level Classification of Skin Cancer with Deep Neural Networks,” Nature, vol. 542, no. 7639, pp. 115–118, 2017. [Google Scholar] [Crossref]

15. International Skin Imaging Collaboration (ISIC), “ISIC Archive,” Available: https://www.isic-archive.com [Google Scholar] [Crossref]

16. Roboflow Inc., “Roboflow Documentation,” Available: https://docs.roboflow.com [Google Scholar] [Crossref]

17. Telegram, “Telegram Bot API Documentation,” Available: https://core.telegram.org/bots/api [Google Scholar] [Crossref]

18. S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, 2017. [Google Scholar] [Crossref]

19. K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” CVPR, pp. 770–778, 2016. [Google Scholar] [Crossref]

20. G. Huang, Z. Liu, L. Van Der Maaten, and K. Weinberger, “Densely Connected Convolutional Networks,” CVPR, pp. 4700–4708, 2017. [Google Scholar] [Crossref]

• The Role of Artificial Intelligence in Revolutionizing Library Services in Nairobi: Ethical Implications and Future Trends in User Interaction
• ESPYREAL: A Mobile Based Multi-Currency Identifier for Visually Impaired Individuals Using Convolutional Neural Network
• Comparative Analysis of AI-Driven IoT-Based Smart Agriculture Platforms with Blockchain-Enabled Marketplaces
• AI-Based Dish Recommender System for Reducing Fruit Waste through Spoilage Detection and Ripeness Assessment
• SEA-TALK: An AI-Powered Voice Translator and Southeast Asian Dialects Recognition