A Comprehensive Comparative Study of Classification and Regression Architectures: Empirical Performance Benchmarking on Standardized Datasets

Supervised learning remains the backbone of predictive analytics. However, the decision to treat a target variable as continuous (Regression) or categorical (Classification) significantly alters model behavior and utility. This paper provides an exhaustive comparison of five classification and five regression techniques. Using the Wine Quality Dataset, we apply identical feature engineering to both paradigms. We measure performance through Mean Squared Error ($MSE$), $R^2$, Accuracy, and F1-Score. The results demonstrate that ensemble methods, specifically Random Forest and XGBoost, consistently outperform linear and kernel-based models, though classification provides a more robust framework for noisy data environments.

Supervised learning, regression, classification

1. Breiman, L. (2001). "Random Forests." Machine Learning, 45(1), 5-32. [Google Scholar] [Crossref]

2. Cortez, P. et al. (2009). "Modeling wine preferences by data mining." Decision Support Systems. [Google Scholar] [Crossref]

3. Pedregosa, F. et al. (2011). "Scikit-learn: Machine Learning in Python." JMLR. [Google Scholar] [Crossref]

4. UCI Machine Learning Repository. Wine Quality Dataset (2026 accessed). [Google Scholar] [Crossref]

• What the Desert Fathers Teach Data Scientists: Ancient Ascetic Principles for Ethical Machine-Learning Practice
• Comparative Analysis of Some Machine Learning Algorithms for the Classification of Ransomware
• Comparative Performance Analysis of Some Priority Queue Variants in Dijkstra’s Algorithm
• Transfer Learning in Detecting E-Assessment Malpractice from a Proctored Video Recordings.
• Dual-Modal Detection of Parkinson’s Disease: A Clinical Framework and Deep Learning Approach Using NeuroParkNet