The Next Generation of Malware Defense using Deep Hashing and Neural Embeddings

The proliferation of polymorphic and metamorphic malware has largely rendered traditional cryptographic signature-based detection ineffective, driving the adoption of similarity-based approaches. This survey systematically examines the evolution of binary similarity detection, tracing the trajectory from classical fuzzy hashing techniques—including ssdeep, sdhash, and TLSH—to contemporary deep representation learning architectures. We analyze state-of-the-art deep hashing methodologies, covering image-based representations via Convolutional Neural Networks (CNNs), structural control-flow graph modeling via Graph Neural Networks (GNNs), and assembly-level semantic analysis using Transformer architectures such as MalBERT and KEENHash. Furthermore, we critically assess the adversarial robustness of these embedding spaces across feature-space and problem-space threat models. By synthesizing recent theoretical developments and empirical benchmarks, this paper provides a unified taxonomy and outlines key challenges for resilient malware lineage tracking and zero-day threat detection.

Malware Defense / Detection; Deep Hashing; Neural Embeddings; Binary Similarity Detection; Fuzzy Hashing; Convolutional Neural Networks (CNNs)

1. Berrios, S., Leiva, D., Olivares, B., Allende-Cid, H., & Hermosilla, P. (2025). Systematic Review: Malware Detection and Classification in Cybersecurity. Applied Sciences, 15(14), 7747. [Google Scholar] [Crossref]

2. https://doi.org/10.3390/app15147747 [Google Scholar] [Crossref]

3. Manjunatha, Vikram & Ramesh, Raghavendra. (2023). Machine Learning in Malware Detection: A Survey of Analysis Techniques. IJARCCE. 12. 204. 10.17148/IJARCCE.2023.12435. [Google Scholar] [Crossref]

4. Frieder Uhlig, Lukas Struppek, Dominik Hintersdorf, Thomas Göbel, Harald Baier, Kristian Kersting. Combining AI and AM - Improving Approximate Matching through Transformer Networks. arXiv:2208.11367v3 [cs.CR] 27 Apr 2023. [Google Scholar] [Crossref]

5. Fuzzy hashing - Wikipedia, accessed November 23, 2025, https://en.wikipedia.org/wiki/Fuzzy_hashing [Google Scholar] [Crossref]

6. Zhijie Liu, Qiyi Tang, Sen Nie, Shi Wu, Liang Feng Zhang, Yutian Tang . KEENHash: Hashing Programs into Function-Aware Embeddings for Large-Scale Binary Code Similarity Analysis, https://arxiv.org/abs/2506.11612 [Google Scholar] [Crossref]

7. Threat Attribution using ssdeep. Medium, accessed November 23, 2025, [Google Scholar] [Crossref]

8. https://nikhilh20.medium.com/fuzzy-hashing-ssdeep-3cade6931b72 [Google Scholar] [Crossref]

9. Ssdeep xref - National Security Agency, accessed August 10, 2026, [Google Scholar] [Crossref]

10. https://code.nsa.gov/emissary/xref/emissary/kff/Ssdeep.html [Google Scholar] [Crossref]

11. Pure Perl ssdeep (CTPH) fuzzy hashing - Ubuntu Manpage, accessed August 10, 2026, [Google Scholar] [Crossref]

12. https://manpages.ubuntu.com/manpages/jammy/man3/Digest::ssdeep.3pm.html [Google Scholar] [Crossref]

13. Fuzzy Hashing Techniques in Applied Malware Analysis - Software Engineering Institute, accessed November 23, 2025, https://www.sei.cmu.edu/blog/fuzzy-hashing-techniques-in-applied-malware-analysis/ [Google Scholar] [Crossref]

14. Oliver, Jonathan & Cheng, Chun & Chen, Yanggui. (2013). TLSH -- A Locality Sensitive Hash. Proceedings - 4th Cybercrime and Trustworthy Computing Workshop, CTC 2013. 7-13. [Google Scholar] [Crossref]

15. 1109/CTC.2013.9. [Google Scholar] [Crossref]

16. Miguel Martín-Pérez, Ricardo J. Rodríguez, Frank Breitinger, Bringing order to approximate matching: Classification and attacks on similarity digest algorithms, Forensic Science International: Digital Investigation, Volume 36, Supplement, 2021, 301120, ISSN 2666-2817, [Google Scholar] [Crossref]

17. https://doi.org/10.1016/j.fsidi.2021.301120. [Google Scholar] [Crossref]

18. All your hashes are belong to us: An overview of malware hashing algorithms - G DATA, accessed November 25, 2025, https://www.gdatasoftware.com/blog/2021/09/an-overview-of-malware-hashing-algorithms [Google Scholar] [Crossref]

19. Frank Breitinger & Harald Baier & Jesse Beckingham (2012). Security and Implementation Analysis of the Similarity Digest sdhash, [Google Scholar] [Crossref]

20. https://dasec.h-da.de/wp-content/uploads/2012/08/2012_08_Breitinger_NeSeFo.pdf [Google Scholar] [Crossref]

21. sdhash package - github.com/eciavatta/sdhash - Go Packages, accessed November 23, 2025, [Google Scholar] [Crossref]

22. https://pkg.go.dev/github.com/eciavatta/sdhash [Google Scholar] [Crossref]

23. Udbhav Prasad (2025). Evaluating Similariy Digests: A Study of TLSH, ssdeep, and sdhash Against Common File Modifications Traditional hashes miss unknown malware. Similarity digests like TLSH, ssdeep, and sdhash improve detection by comparing file similarities. This article benchmarks them, https://dzone.com/articles/similarity-digests-tlsh-ssdeep-sdhash-benchmark [Google Scholar] [Crossref]

24. Jonathan Oliver, Chun Cheng and Yanggui Chen, TLSH - A Locality Sensitive Hash, Trend Micro, https://documents.trendmicro.com/assets/wp/wp-locality-sensitive-hash.pdf [Google Scholar] [Crossref]

25. Liu H., Hagen J., Ali M. and Oliver J. (2023). An Evaluation of Malware Triage Similarity Hashes. In Proceedings of the 25th International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-648-4, SciTePress, pages 431-435. DOI: 10.5220/0011728500003467 [Google Scholar] [Crossref]

26. Jonathan Oliver and Josiah Hagen (2021). Designing the Elements of a Fuzzy Hashing Scheme, 2021 IEEE 19th International Conference on Embedded and Ubiquitous Computing (EUC) Pages: 1–6, DOI: 10.1109/euc53437.2021.00028 [Google Scholar] [Crossref]

27. Matteo Brosolo, Asmitha K. A., Mauro Conti, Rafidha Rehiman K. A., Muhammed Shafi K. P., Serena Nicolazzo, Antonino Nocera, Vinod P. Security through the Eyes of AI: How Visualization is Shaping Malware Detection, https://arxiv.org/pdf/2505.07574 [Google Scholar] [Crossref]

28. N. Sai Ramana Vashista and K. Abhimanyu Kumar Patro, "Enhancing Malware Analysis Using Data Visualization Through Shared Code and Attribute Analysis," in IEEE Access, vol. 13, pp. 107482-107498, 2025, doi: 10.1109/ACCESS.2025.3582164. [Google Scholar] [Crossref]

29. D. Kim, J. Hur and M. Yoon, "Scalable and Multifaceted Search and Its Application for Binary Malware Files," in IEEE Access, vol. 9, pp. 112770-112779, 2021, doi: [Google Scholar] [Crossref]

30. 1109/ACCESS.2021.3102157. [Google Scholar] [Crossref]

31. S. J. I. Ismail, Hendrawan, B. Rahardjo, T. Juhana and Y. Musashi, "MalSSL—Self-Supervised Learning for Accurate and Label-Efficient Malware Classification," in IEEE Access, vol. 12, pp. [Google Scholar] [Crossref]

32. 58823-58835, 2024, doi: 10.1109/ACCESS.2024.3392251. [Google Scholar] [Crossref]

33. Zhang, Yunchun & Liao, Zikun & Zhang, Ning & Min, Shaohui & Wang, Qi & Quek, Tony Q.S. & Zhao, Mingxiong. (2024). Deep Hashing for Malware Family Classification and New Malware Identification. IEEE Internet of Things Journal. 11. 26837-26851. 10.1109/JIOT.2024.3353250. [Google Scholar] [Crossref]

34. Chen, Y.-H., Chen, J.-L., & Deng, R.-F. (2022). Similarity-Based Malware Classification Using Graph Neural Networks. Applied Sciences, 12(21), 10837. https://doi.org/10.3390/app122110837 [Google Scholar] [Crossref]

35. Guo, W., Du, W., Yang, X., Xue, J., Wang, Y., Han, W., & Hu, J. (2025). MalHAPGNN: An Enhanced Call Graph-Based Malware Detection Framework Using Hierarchical Attention Pooling Graph Neural Network. Sensors, 25(2), 374. https://doi.org/10.3390/s25020374 [Google Scholar] [Crossref]

36. Tristan Bilot, Nour El Madhoun, Khaldoun Al Agha, Anis Zouaoui. A Survey on Malware Detection with Graph Representation Learning, https://arxiv.org/pdf/2303.16004 [Google Scholar] [Crossref]

37. Z. H. Qaisar, S. H. Almotiri, M. A. Al Ghamdi, A. A. Nagra and G. Ali, "A Scalable and Efficient Multi-Agent Architecture for Malware Protection in Data Sharing Over Mobile Cloud," in IEEE Access, vol. 9, pp. 76248-76259, 2021, doi: 10.1109/ACCESS.2021.3067284. [Google Scholar] [Crossref]

38. Detection of Prevalent Malware Families with Deep Learning - Microsoft, accessed November 23, 2025, [Google Scholar] [Crossref]

39. https://www.microsoft.com/en-us/research/wp-content/uploads/2020/07/Siamese_Milcom2019.pdf [Google Scholar] [Crossref]

40. Kartikeya Aneja, Nagender Aneja, Murat Kantarcioglu, Learning Joint Embeddings of Function and Process Call Graphs for Malware Detection, https://arxiv.org/html/2510.09984v1 [Google Scholar] [Crossref]

41. Luca Massarelli, Giuseppe Antonio Di Luna, Fabio Petroni, Leonardo Querzoni, Roberto Baldoni, SAFE: Self-Attentive Function Embeddings for Binary Similarity, arXiv:1811.05296 [Google Scholar] [Crossref]

42. Jia Y, Yu Z, Hong Z. Semantic aware-based instruction embedding for binary code similarity detection. PLoS One. 2024 Jun 11;19(6):e0305299. doi: 10.1371/journal.pone.0305299. PMID: 38861533; PMCID: PMC11166306. [Google Scholar] [Crossref]

43. Luca Massarelli, SAFE: a step into the creation of embeddings for binary code similarity detection, Medium, accessed November 24, 2025, https://medium.com/@massarelli/safe-self-attentive-function-embedding-d80abbfea794 [Google Scholar] [Crossref]

44. Rahali, A., & Akhloufi, M. A. (2023). MalBERTv2: Code Aware BERT-Based Model for Malware Identification. Big Data and Cognitive Computing, 7(2), 60. https://doi.org/10.3390/bdcc7020060 [Google Scholar] [Crossref]

45. Pascal Maniriho, Abdun Naser Mahmood, Mohammad Jabed Morshed Chowdhury. EarlyMalDetect: A Novel Approach for Early Windows Malware Detection Based on Sequences of API Calls, https://arxiv.org/html/2407.13355v1 [Google Scholar] [Crossref]

46. Minghao Hu, Junzhe Wang, Weisen Zhao, Qiang Zeng, Lannan Luo, FlowMalTrans: Unsupervised Binary Code Translation for Malware Detection Using Flow-Adapter Architecture, https://arxiv.org/html/2508.20212v1 [Google Scholar] [Crossref]

47. Kshitiz Aryal, Maanak Gupta, Mahmoud Abdelsalam, Moustafa Saleh, Explainability Guided Adversarial Evasion Attacks on Malware Detectors, https://arxiv.org/html/2405.01728v1 [Google Scholar] [Crossref]

48. Bojan Kolosnjaji, Ambra Demontis, Battista Biggio, Davide Maiorca, Giorgio Giacinto, Claudia Eckert, Fabio Roli, Adversarial Malware Binaries: Evading Deep Learning for Malware Detection in Executables, https://arxiv.org/abs/1803.04173 [Google Scholar] [Crossref]

49. Pavla Louthánová, Matouš Kozák, Martin Jureček, Mark Stamp, and Fabio Di Troia. "A comparison of adversarial malware generators" Journal of Computer Virology and Hacking Techniques (2024). https://doi.org/10.1007/s11416-024-00519-z [Google Scholar] [Crossref]

50. Kulkarni, S. S., & Di Troia, F. (2025). Robust Hashing for Improved CNN Performance in Image-Based Malware Detection. Electronics, 14(19), 3915. https://doi.org/10.3390/electronics14193915 [Google Scholar] [Crossref]

51. CrowdStrike, Researchers Explore Contrastive Learning for Malware Detection, accessed November 23, 2025, [Google Scholar] [Crossref]

52. https://www.crowdstrike.com/en-us/blog/contrastive-learning-enhance-malware-threat-detection/ [Google Scholar] [Crossref]

53. Alsubaei, Faisal & Almazroi, Abdulwahab & Atwa, Walid & Almazroi, Abdulaleem & Ayub, Nasir & Jhanjhi, Noor. (2025). Adaptive malware identification via integrated SimCLR and GRU networks. Scientific Reports. 15. 10.1038/s41598-025-08556-4. [Google Scholar] [Crossref]

54. Kapoor, G., Nadipalli, S., & Di Troia, F. (2025). Embedding-Driven Synthetic Malware Generation with Autoencoders and Cluster-Tangent Diffusion. Applied Sciences, 15(21), 11791. [Google Scholar] [Crossref]

55. https://doi.org/10.3390/app152111791 [Google Scholar] [Crossref]

56. Mostafa Jafari, Alireza Shameli-Sendi, Evaluating the robustness of adversarial defenses in malware detection systems, https://arxiv.org/html/2505.09342v1 [Google Scholar] [Crossref]

57. Rahali, Abir & Akhloufi, Moulay. (2023). MalBERTv2: Code Aware BERT-Based Model for Malware Identification. Big Data and Cognitive Computing. 7. 60. 10.3390/bdcc7020060. [Google Scholar] [Crossref]

• What the Desert Fathers Teach Data Scientists: Ancient Ascetic Principles for Ethical Machine-Learning Practice
• Comparative Analysis of Some Machine Learning Algorithms for the Classification of Ransomware
• Comparative Performance Analysis of Some Priority Queue Variants in Dijkstra’s Algorithm
• Transfer Learning in Detecting E-Assessment Malpractice from a Proctored Video Recordings.
• Dual-Modal Detection of Parkinson’s Disease: A Clinical Framework and Deep Learning Approach Using NeuroParkNet