Heuristic-Based Approaches in Fuzzy Clustering: A Comprehensive Review

Fuzzy clustering has emerged as a powerful technique for analyzing complex, uncertain, and high-dimensional data across diverse application domains, including pattern recognition, bioinformatics, image analysis, and decision support systems. Unlike classical clustering, which assigns each data instance to a single cluster, fuzzy clustering allows partial membership, thereby capturing inherent ambiguity in real-world datasets. This review provides a comprehensive examination of heuristic-based fuzzy clustering algorithms. We begin by outlining the fundamental concepts of clustering, fuzzy set theory, and the principles of fuzzy clustering. Subsequently, we discuss the evolution of core algorithms, including Fuzzy C-Means (FCM) and Possibilistic C-Means (PCM), and highlight significant modifications derived from altering distance metrics, objective functions, and optimization strategies. Particular emphasis is placed on heuristic and metaheuristic enhancements—such as genetic algorithms, particle swarm optimization, and artificial immune systems—that address the limitations of classical approaches, including sensitivity to initialization, susceptibility to noise and outliers, and premature convergence. Recent contributions in hybrid fuzzy clustering are also reviewed, with attention to their strengths, weaknesses, and potential applications. Finally, we synthesize insights from the literature to categorize the persistent disadvantages of existing methods and identify promising directions for future research, including adaptive fuzzifiers, noise-resilient models, and integration with evolutionary computation. This study not only consolidates advances in heuristic-based fuzzy clustering but also provides guidance for researchers aiming to design more robust, scalable, and application-driven clustering algorithms.

Data Mining and Heuristic-Based Approaches

1. C. Dunn, "A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters," 1973. [Google Scholar] [Crossref]

2. J. C. Bezdek, Pattern recognition with fuzzy objective function algorithms: Kluwer Academic Publishers, 1981. [Google Scholar] [Crossref]

3. J. C. Bezdek, "Fuzzy Mathematics in Pattern Classification," Applied Math. Center, Cornell University, 1973. [Google Scholar] [Crossref]

4. M.-S. Yang and C.-H. Ko, "On a class of fuzzy c-numbers clustering procedures for fuzzy data," Fuzzy Sets and Systems, vol. 84, pp. 49-60, 11/25/ 1996. [Google Scholar] [Crossref]

5. M. S. Yang, "A survey of fuzzy clustering," Mathematical and Computer Modelling, vol. 18, pp. 1-16, 12// 1993. [Google Scholar] [Crossref]

6. D. E. Gustafson and W. C. Kessel, "Fuzzy clustering with a fuzzy covariance matrix," in Decision and Control including the 17th Symposium on Adaptive Processes, 1978 IEEE Conference on, 1978, pp. 761-766. [Google Scholar] [Crossref]

7. R. Krishnapuram and J. M. Keller, "A possibilistic approach to clustering," Fuzzy Systems, IEEE Transactions on, vol. 1, pp. 98-110, 1993. [Google Scholar] [Crossref]

8. R. N. Dave and R. Krishnapuram, "Robust clustering methods: a unified view," Fuzzy Systems, IEEE Transactions on, vol. 5, pp. 270-293, 1997. [Google Scholar] [Crossref]

9. D. Dubois and H. Prade, Possibility theory: Springer, 1988. [Google Scholar] [Crossref]

10. M. B. Bonab, S. Z. M. Hashim, S. M. Shamsuddin, and N. Gharaei, "A Robust Hyper-Heuristic Algorithm For Clustering," in International Science Postgraduate Conference (ISPC2016), 4th International Conference on, 2016, pp. 515-524. [Google Scholar] [Crossref]

11. M. B. Bonab, "Modified K Modified K-Means Algorithm for Genetic Clustering Means Algorithm for Genetic Clustering Means Algorithm for Genetic Clustering," IJCSNS, vol. 11, p. 24, 2011. [Google Scholar] [Crossref]

12. L. A. Zadeh, "Fuzzy sets," Information and Control, vol. 8, pp. 338-353, 6// 1965. [Google Scholar] [Crossref]

13. L. A. Zadeh, "Outline of a New Approach to the Analysis of Complex Systems and Decision Processes," Systems, Man and Cybernetics, IEEE Transactions on, vol. SMC-3, pp. 28-44, 1973. [Google Scholar] [Crossref]

14. L. A. Zadeh, "Fuzzy algorithms," Information and Control, vol. 12, pp. 94-102, 2// 1968. [Google Scholar] [Crossref]

15. E. T. Lee and L. A. Zadeh, "Note on fuzzy languages," Information Sciences, vol. 1, pp. 421-434, 10// 1969. [Google Scholar] [Crossref]

16. L. A. Zadeh, "The concept of a linguistic variable and its application to approximate reasoning—II," Information Sciences, vol. 8, pp. 301-357, // 1975. [Google Scholar] [Crossref]

17. L. A. Zadeh, "The concept of a linguistic variable and its application to approximate reasoning-III," Information Sciences, vol. 9, pp. 43-80, // 1975. [Google Scholar] [Crossref]

18. L. A. Zadeh, "The concept of a linguistic variable and its application to approximate reasoning—I," Information Sciences, vol. 8, pp. 199-249, // 1975. [Google Scholar] [Crossref]

19. L. A. Zadeh, "Quantitative fuzzy semantics," Information Sciences, vol. 3, pp. 159-176, 4// 1971. [Google Scholar] [Crossref]

20. L. A. Zadeh, "Similarity relations and fuzzy orderings," Information Sciences, vol. 3, pp. 177-200, 4// 1971. [Google Scholar] [Crossref]

21. L. A. Zadeh, "Probability measures of Fuzzy events," Journal of Mathematical Analysis and Applications, vol. 23, pp. 421-427, 8// 1968. [Google Scholar] [Crossref]

22. F. Klawonn and F. Höppner, "What Is Fuzzy about Fuzzy Clustering? Understanding and Improving the Concept of the Fuzzifier," in Advances in Intelligent Data Analysis V. vol. 2810, M. R. Berthold, H.-J. Lenz, E. Bradley, R. Kruse, and C. Borgelt, Eds., ed: Springer Berlin Heidelberg, 2003, pp. 254-264. [Google Scholar] [Crossref]

23. I. Gath and A. B. Geva, "Unsupervised optimal fuzzy clustering," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 11, pp. 773-780, 1989. [Google Scholar] [Crossref]

24. M. Babrdelbonb, S. Z. M. H. M. Hashim, and N. E. N. Bazin, "Data Analysis by Combining the Modified K-Means and Imperialist Competitive Algorithm," Jurnal Teknologi, vol. 70, 2014. [Google Scholar] [Crossref]

25. J. C. Dunn, "A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters," Journal of Cybernetics, vol. 3, pp. 32-57, 1973/01/01 1973. [Google Scholar] [Crossref]

26. J. C. Bezdek†, "Cluster validity with fuzzy sets," 1973. [Google Scholar] [Crossref]

27. R. O. Duda and P. E. Hart, Pattern classification and scene analysis vol. 3: Wiley New York, 1973. [Google Scholar] [Crossref]

28. G. H. Ball and D. J. Hall, "ISODATA, a novel method of data analysis and pattern classification," DTIC Document1965. [Google Scholar] [Crossref]

29. M. B. Bonab, Y. H. Tay, S. Z. Mohd Hashim, and K. T. Soon, "An Efficient Robust Hyper-Heuristic Algorithm to Clustering Problem," Cham, 2019, pp. 48-60. [Google Scholar] [Crossref]

30. D. Dubois and H. Prade, "Possibility theory. 1988," ed: Plenum Press, New York. [Google Scholar] [Crossref]

31. F. Klawonn, R. Kruse, and H. Timm, "Fuzzy Shell Cluster Analysis," in Learning, Networks and Statistics. vol. 382, G. Della Riccia, H.-J. Lenz, and R. Kruse, Eds., ed: Springer Vienna, 1997, pp. 105-119. [Google Scholar] [Crossref]

32. M. Yaghini, M. Ranjpour, and F. Yousefi, "A Survey of Fuzzy Clustering Algorithms," presented at the The 3rd Iran Data Mining Conference, 2009. [Google Scholar] [Crossref]

33. J. Keller, R. Krisnapuram, and N. R. Pal, Fuzzy models and algorithms for pattern recognition and image processing vol. 4: Springer, 2005. [Google Scholar] [Crossref]

34. J. C. Bezdek and S. K. Pal, Fuzzy models for pattern recognition vol. 56: IEEE Press, New York, 1992. [Google Scholar] [Crossref]

35. Z. Chi, H. Yan, and T. Pham, Fuzzy algorithms: with applications to image processing and pattern recognition vol. 10: World Scientific, 1996. [Google Scholar] [Crossref]

36. V. Vapnik, The nature of statistical learning theory: springer, 2000. [Google Scholar] [Crossref]

37. V. N. Vapnik, The nature of statistical learning theory: Springer-Verlag New York, Inc., 1995. [Google Scholar] [Crossref]

38. B. Schölkopf and A. J. Smola, Learning with kernels: support vector machines, regularization, optimization, and beyond: MIT press, 2002. [Google Scholar] [Crossref]

39. Z.-d. Wu, W.-x. Xie, and J.-p. Yu, "Fuzzy c-means clustering algorithm based on kernel method," in Computational Intelligence and Multimedia Applications, 2003. ICCIMA 2003. Proceedings. Fifth International Conference on, 2003, pp. 49-54. [Google Scholar] [Crossref]

40. D.-Q. Zhang and S.-C. Chen, "Clustering Incomplete Data Using Kernel-Based Fuzzy C-means Algorithm," Neural Processing Letters, vol. 18, pp. 155-162, 2003/12/01 2003. [Google Scholar] [Crossref]

41. D. Q. Zhang and S. C. Chen, "Kernel-based fuzzy and possibilistic c-means clustering," in Proceedings of the International Conference Artificial Neural Network, 2003, pp. 122-125. [Google Scholar] [Crossref]

42. R. N. Dave, "Characterization and detection of noise in clustering," Pattern Recognition Letters, vol. 12, pp. 657-664, 11// 1991. [Google Scholar] [Crossref]

43. N. N. Davé and S. Sen, "On generalizing the noise clustering algorithms," Proc. of The 7th IFSA world congress, vol. 3, pp. 205-210, 1997. [Google Scholar] [Crossref]

44. R. Dave and S. Sen, "Generalized noise clustering as a robust fuzzy cm-estimators model," in Fuzzy Information Processing Society-NAFIPS, 1998 Conference of the North American, 1998, pp. 256-260. [Google Scholar] [Crossref]

45. H. Frigui and R. Krishnapuram, "A robust algorithm for automatic extraction of an unknown number of clusters from noisy data," Pattern Recognition Letters, vol. 17, pp. 1223-1232, 10/25/ 1996. [Google Scholar] [Crossref]

46. A. Keller, "Fuzzy clustering with outliers," in Fuzzy Information Processing Society, 2000. NAFIPS. 19th International Conference of the North American, 2000, pp. 143-147. [Google Scholar] [Crossref]

47. F. Klawonn and F. Höppner, "An alternative approach to the fuzzifier in fuzzy clustering to obtain better clustering results," in Proc. 3rd Eusflat Conference, 2003, pp. 730-734. [Google Scholar] [Crossref]

48. F. Klawonn and F. Höppner, "What is fuzzy about fuzzy clustering? Understanding and improving the concept of the fuzzifier," in Advances in Intelligent Data Analysis V, ed: Springer, 2003, pp. 254-264. [Google Scholar] [Crossref]

49. P. J. Rousseeuw, E. Trauwaert, and L. Kaufman, "Fuzzy clustering with high contrast," Journal of Computational and Applied Mathematics, vol. 64, pp. 81-90, 11/30/ 1995. [Google Scholar] [Crossref]

50. F. Klawonn, "Fuzzy clustering: insights and a new approach," Mathware & soft computing, vol. 11, pp. 125-142, 2008. [Google Scholar] [Crossref]

51. H. Sahbi and B. Nozha, "Validity of Fuzzy Clustering Using Entropy Regularization," in Fuzzy Systems, 2005. FUZZ '05. The 14th IEEE International Conference on, 2005, pp. 177-182. [Google Scholar] [Crossref]

52. H. Frigui and R. Krishnapuram, "Clustering by competitive agglomeration," Pattern Recognition, vol. 30, pp. 1109-1119, 7// 1997. [Google Scholar] [Crossref]

53. H. Frigui and R. Krishnapuram, "A robust competitive clustering algorithm with applications in computer vision," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 21, pp. 450-465, 1999. [Google Scholar] [Crossref]

54. M. Halkidi, Y. Batistakis, and M. Vazirgiannis, "Clustering validity checking methods: part II," SIGMOD Rec., vol. 31, pp. 19-27, 2002. [Google Scholar] [Crossref]

55. M. Halkidi, Y. Batistakis, and M. Vazirgiannis, "Cluster validity methods: part I," SIGMOD Rec., vol. 31, pp. 40-45, 2002. [Google Scholar] [Crossref]

56. H. Timm and R. Kruse, "A modification to improve possibilistic fuzzy cluster analysis," in Fuzzy Systems, 2002. FUZZ-IEEE'02. Proceedings of the 2002 IEEE International Conference on, 2002, pp. 1460-1465. [Google Scholar] [Crossref]

57. H. Timm, C. Borgelt, C. Döring, and R. Kruse, "An extension to possibilistic fuzzy cluster analysis," Fuzzy Sets and Systems, vol. 147, pp. 3-16, 10/1/ 2004. [Google Scholar] [Crossref]

58. N. R. Pal, K. Pal, and J. C. Bezdek, "A mixed c-means clustering model," in Fuzzy Systems, 1997., Proceedings of the Sixth IEEE International Conference on, 1997, pp. 11-21 vol.1. [Google Scholar] [Crossref]

59. N. R. Pal, K. Pal, J. M. Keller, and J. C. Bezdek, "A new hybrid c-means clustering model," in Fuzzy Systems, 2004. Proceedings. 2004 IEEE International Conference on, 2004, pp. 179-184 vol.1. [Google Scholar] [Crossref]

60. R. N. Dave and S. Sen, "Generalized noise clustering as a robust fuzzy c-M-estimators model," in Fuzzy Information Processing Society - NAFIPS, 1998 Conference of the North American, 1998, pp. 256-260. [Google Scholar] [Crossref]

61. E. R. Hruschka, R. J. G. B. Campello, and L. N. de Castro, "Evolutionary search for optimal fuzzy c-means clustering," in Fuzzy Systems, 2004. Proceedings. 2004 IEEE International Conference on, 2004, pp. 685-690 vol.2. [Google Scholar] [Crossref]

62. D. Yun-Ying, Z. Yun-Jie, and C. Chun-Ling, "Multistage random sampling genetic-algorithm-based fuzzy c-means clustering algorithm," in Machine Learning and Cybernetics, 2004. Proceedings of 2004 International Conference on, 2004, pp. 2069-2073 vol.4. [Google Scholar] [Crossref]

63. N. R. Pal, K. Pal, J. M. Keller, and J. C. Bezdek, "A Possibilistic Fuzzy c-Means Clustering Algorithm," Fuzzy Systems, IEEE Transactions on, vol. 13, pp. 517-530, 2005. [Google Scholar] [Crossref]

64. L. O. Hall and P. M. Kanade, "Swarm Based Fuzzy Clustering with Partition Validity," in Fuzzy Systems, 2005. FUZZ '05. The 14th IEEE International Conference on, 2005, pp. 991-995. [Google Scholar] [Crossref]

65. W. Xiao-Hong and Z. Jian-Jiang, "Possibilistic Fuzzy c-Means Clustering Model Using Kernel Methods," in Computational Intelligence for Modelling, Control and Automation, 2005 and International Conference on Intelligent Agents, Web Technologies and Internet Commerce, International Conference on, 2005, pp. 465-470. [Google Scholar] [Crossref]

66. L. Li and X. Wenbo, "An Immune-Inspired Evolutionary Fuzzy Clustering Algorithm Based on Constrained Optimization," in Intelligent Systems Design and Applications, 2006. ISDA '06. Sixth International Conference on, 2006, pp. 966-970. [Google Scholar] [Crossref]

67. B. Ojeda-Magafia, R. Ruelas, M. Corona-Nakamura, and D. Andina, "An improvement to the possibilistic fuzzy c-means clustering algorithm," in Automation Congress, 2006. WAC'06. World, 2006, pp. 1-8. [Google Scholar] [Crossref]

68. I. Ohyama, Y. Suzuki, S. Saga, and J. Maeda, "Fuzzy Modeling Based on Noise Cluster and Possibilistic Clustering," in Adaptive and Learning Systems, 2006 IEEE Mountain Workshop on, 2006, pp. 225-230. [Google Scholar] [Crossref]

69. S. Miyamoto, R. Inokuchi, and Y. Kuroda, "Possibilistic and Fuzzy c-Means Clustering with Weighted Objects," in Fuzzy Systems, 2006 IEEE International Conference on, 2006, pp. 869-874. [Google Scholar] [Crossref]

70. F. Masulli and S. Rovetta, "Soft transition from probabilistic to possibilistic fuzzy clustering," Fuzzy Systems, IEEE Transactions on, vol. 14, pp. 516-527, 2006. [Google Scholar] [Crossref]

71. Q. Jiang and M. Jia, "Novel Hybrid Clustering Algorithm Incorporating Artificial Immunity into Fuzzy Kernel Clustering for Pattern Recognition," in Control Conference, 2007. CCC 2007. Chinese, 2007, pp. 592-596. [Google Scholar] [Crossref]

72. P. Maji and S. K. Pal, "Rough Set Based Generalized Fuzzy <formula formulatype="inline"> <img src="/images/tex/276.gif" alt="C"> </formula> -Means Algorithm and Quantitative Indices," Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, vol. 37, pp. 1529-1540, 2007. [Google Scholar] [Crossref]

73. W. Hao, Y. Shiqin, X. Wenbo, and S. Jun, "Scalability of Hybrid Fuzzy C-Means Algorithm Based on Quantum-Behaved PSO," in Fuzzy Systems and Knowledge Discovery, 2007. FSKD 2007. Fourth International Conference on, 2007, pp. 261-265. [Google Scholar] [Crossref]

74. C. Wei and F. Kangling, "A hybridized clustering approach using particle swarm optimization for image segmentation," in Audio, Language and Image Processing, 2008. ICALIP 2008. International Conference on, 2008, pp. 1365-1368. [Google Scholar] [Crossref]

75. S. Di Nuovo and V. Catania, "An evolutionary fuzzy c-means approach for clustering of bio-informatics databases," in Fuzzy Systems, 2008. FUZZ-IEEE 2008. (IEEE World Congress on Computational Intelligence). IEEE International Conference on, 2008, pp. 2077-2082. [Google Scholar] [Crossref]

76. Y. Zhou, Y. e. Li, and S. Xia, "Robust Fuzzy-Possibilistic C-Means Algorithm," in Intelligent Information Technology Application, 2008. IITA '08. Second International Symposium on, 2008, pp. 669-673. [Google Scholar] [Crossref]

77. M.-S. Yang and H.-S. Tsai, "A Gaussian kernel-based fuzzy c-means algorithm with a spatial bias correction," Pattern Recognition Letters, vol. 29, pp. 1713-1725, 9/1/ 2008. [Google Scholar] [Crossref]

78. S.-h. Liu and H.-f. Hou, "A combination of mixture Genetic Algorithm and Fuzzy C-means Clustering Algorithm," in IT in Medicine & Education, 2009. ITIME '09. IEEE International Symposium on, 2009, pp. 254-258. [Google Scholar] [Crossref]

79. Szila, x, L. gyi, D. Iclanzan, S. M. Szilagyi, D. Dumitrescu, et al., "A generalized c-means clustering model using optimized via evolutionary computation," in Fuzzy Systems, 2009. FUZZ-IEEE 2009. IEEE International Conference on, 2009, pp. 451-455. [Google Scholar] [Crossref]

80. Z. Wei, L. Cheng, and Z. Yu-zhu, "A new hybrid algorithm for image segmentation based on rough sets and enhanced fuzzy c-means clustering," in Automation and Logistics, 2009. ICAL '09. IEEE International Conference on, 2009, pp. 1212-1216. [Google Scholar] [Crossref]

81. Szila, x, L. gyi, S. M. Szilagyi, and Z. Benyo, "A unified approach to c-means clustering models," in Fuzzy Systems, 2009. FUZZ-IEEE 2009. IEEE International Conference on, 2009, pp. 456-461. [Google Scholar] [Crossref]

82. H. Izakian, A. Abraham, and V. Snasel, "Fuzzy clustering using hybrid fuzzy c-means and fuzzy particle swarm optimization," in Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on, 2009, pp. 1690-1694. [Google Scholar] [Crossref]

83. C. Yongming, J. Mingyan, and Y. Dongfeng, "Novel Clustering Algorithms Based on Improved Artificial Fish Swarm Algorithm," in Fuzzy Systems and Knowledge Discovery, 2009. FSKD '09. Sixth International Conference on, 2009, pp. 141-145. [Google Scholar] [Crossref]

84. Q. Fuheng, M. SiLiang, and H. Yating, "Generalized Possibilistic C-Means Clustering Based on Differential Evolution Algorithm," in Intelligent Systems and Applications, 2009. ISA 2009. International Workshop on, 2009, pp. 1-4. [Google Scholar] [Crossref]

85. H. Dong, Y. Dong, C. Zhou, G. Yin, and W. Hou, "A fuzzy clustering algorithm based on evolutionary programming," Expert Systems with Applications, vol. 36, pp. 11792-11800, 11// 2009. [Google Scholar] [Crossref]

86. U. Maulik and I. Saha, "Modified differential evolution based fuzzy clustering for pixel classification in remote sensing imagery," Pattern Recognition, vol. 42, pp. 2135-2149, 9// 2009. [Google Scholar] [Crossref]

87. W. Hao, L. Danyun, and C. Yayun, "A New Scalability of Hybrid Fuzzy C-Means Algorithm," in Artificial Intelligence and Computational Intelligence (AICI), 2010 International Conference on, 2010, pp. 55-58. [Google Scholar] [Crossref]

88. L. Zhu, S. Qu, and T. Du, "Adaptive fuzzy clustering based on Genetic algorithm," in Advanced Computer Control (ICACC), 2010 2nd International Conference on, 2010, pp. 79-82. [Google Scholar] [Crossref]

• Assessment of the Role of Artificial Intelligence in Repositioning TVET for Economic Development in Nigeria
• Teachers’ Use of Assure Model Instructional Design on Learners’ Problem Solving Efficacy in Secondary Schools in Bungoma County, Kenya
• “E-Booksan Ang Kaalaman”: Development, Validation, and Utilization of Electronic Book in Academic Performance of Grade 9 Students in Social Studies
• Analyzing EFL University Students’ Academic Speaking Skills Through Self-Recorded Video Presentation
• Major Findings of The Study on Total Quality Management in Teachers’ Education Institutions (TEIs) In Assam – An Evaluative Study