Synthetic Data Quality Evaluation in Generative AI: Current Trends, Challenges, and Future Directions for Social Science Research

Generative Artificial Intelligence (GenAI) has transformed data generation through advanced models such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), Diffusion Models, and Large Language Models (LLMs), enabling the creation of synthetic datasets that closely resemble real-world data while addressing challenges related to privacy, accessibility, and regulatory compliance. As synthetic data becomes increasingly adopted across healthcare, education, finance, public administration, and social science research, ensuring its quality, reliability, fairness, and trustworthiness has emerged as a critical research priority. This literature review examines recent developments in synthetic data quality evaluation between 2020 and 2026, focusing on key dimensions including utility, fidelity, privacy preservation, fairness, diversity, robustness, interpretability, and governance. The review traces the evolution of evaluation methodologies from traditional statistical similarity measures toward multidimensional assessment frameworks such as SynEval, SynthEval, Benchmarking Synthetic Tabular Data Framework, SynAE, and ESDAE. A structured comparison of these frameworks is presented using criteria including analytical accuracy, scalability, privacy protection, fairness assessment, and interpretability. The review further explores emerging approaches for explainable synthetic data assessment, fairness-aware synthetic data generation, and Privacy-Enhancing Technologies (PETs), including differential privacy, federated learning, secure multi-party computation, and privacy-preserving generative models. In addition, real-world case studies from healthcare, education, public policy, and social science research are examined to demonstrate how synthetic data quality evaluation directly influences decision-making, research validity, and policy outcomes. The findings indicate that while modern generative models can produce highly realistic and analytically useful datasets, persistent challenges remain, including the lack of standardized benchmarking protocols, utility–privacy trade-offs, privacy leakage risks, bias amplification, limited explainability, and governance concerns. The review concludes that future research should prioritize internationally accepted evaluation standards, explainable and fairness-aware assessment frameworks, stronger privacy-preserving mechanisms, and comprehensive governance models to support the responsible, transparent, and trustworthy deployment of synthetic data in the Generative AI era.

Synthetic Data, Generative Artificial Intelligence, Data Quality Evaluation, Utility, Fidelity, Privacy Preservation, Fairness, Explainable AI, Privacy-Enhancing Technologies, Social Science Research

1. El Emam, K., Mosquera, L., & Hoptroff, R. (2020). Practical synthetic data generation: Balancing privacy and the broad availability of data. O'Reilly Media. [Google Scholar] [Crossref]

2. Tucker, A., Wang, Z., Rotalinti, Y., & Myles, P. (2020). Generating high-fidelity synthetic health data for assessing machine learning healthcare software. npj Digital Medicine, 3(1), 147. https://doi.org/10.1038/s41746-020-00353-9 [Google Scholar] [Crossref]

3. Goncalves, A., Ray, P., Soper, B., Stevens, J., Coyle, L., & Sales, A. P. (2020). Generation and evaluation of synthetic patient data. BMC Medical Research Methodology, 20(1), 108. https://doi.org/10.1186/s12874-020-00977-1 [Google Scholar] [Crossref]

4. Goyal, M., & Mahmoud, Q. H. (2024). A systematic review of synthetic data generation techniques using generative AI. Electronics, 13(17), 3509. https://doi.org/10.3390/electronics13173509 [Google Scholar] [Crossref]

5. Bauer, A., Trapp, S., Stenger, M., Leppich, R., Kounev, S., Leznik, M., Chard, K., & Foster, I. (2024). Comprehensive exploration of synthetic data generation: A survey. [Google Scholar] [Crossref]

6. Long, L., Wang, R., Xiao, R., Zhao, J., Ding, X., Chen, G., & Wang, H. (2024). On LLMs-driven synthetic data generation, curation, and evaluation: A survey. Findings of the Association for Computational Linguistics: ACL 2024, 11065–11082. https://doi.org/10.18653/v1/2024.findings-acl.658 [Google Scholar] [Crossref]

7. Livieris, I. E., Alimpertis, N., Domalis, G., & Tsakalidis, D. (2024). An evaluation framework for synthetic data generation models. In Artificial Intelligence Applications and Innovations (pp. 287–299). Springer. https://doi.org/10.1007/978-3-031-63219-8_24 [Google Scholar] [Crossref]

8. Yuan, Y., Liu, Y., & Cheng, L. (2024). A multi-faceted evaluation framework for assessing synthetic data generated by large language models. arXiv. https://arxiv.org/abs/2404.14445 [Google Scholar] [Crossref]

9. Iskander, S., Tolmach, S., Shapira, O., Cohen, N., & Karnin, Z. (2024). Quality matters: Evaluating synthetic data for tool-using LLMs. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 4958–4976. https://doi.org/10.18653/v1/2024.emnlp-main.285 [Google Scholar] [Crossref]

10. Bellamy, R. K. E., Dey, K., Hind, M., et al. (2019). AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 63(4/5), 4:1–4:15. [Google Scholar] [Crossref]

11. Stadler, T., Oprisanu, B., & Troncoso, C. (2022). Synthetic data: Anonymisation ground truth. Proceedings on Privacy Enhancing Technologies, 2022(1), 499–527. [Google Scholar] [Crossref]

12. Steier, A., Ramaswamy, L., Manoel, A., & Haushalter, A. (2025). Synthetic data privacy metrics. arXiv. https://arxiv.org/abs/2501.03941 [Google Scholar] [Crossref]

13. Schlegel, V., Bharath, A. A., Zhao, Z., & Yee, K. (2025). Generating synthetic data with formal privacy guarantees: State of the art and the road ahead. arXiv. https://arxiv.org/abs/2503.20846 [Google Scholar] [Crossref]

14. Sidorenko, A., Platzer, M., Scriminaci, M., & Tiwald, P. (2025). Benchmarking synthetic tabular data: A multi-dimensional evaluation framework. arXiv. https://arxiv.org/abs/2504.01908 [Google Scholar] [Crossref]

15. Mönchmeyer, R., et al. (2024). Synthetic data in biomedicine via generative artificial intelligence. Nature Reviews Bioengineering, 2(10), 713–731. https://doi.org/10.1038/s44222-024-00245-7 [Google Scholar] [Crossref]

16. World Economic Forum. (2025). Synthetic data: The new data frontier. World Economic Forum [Google Scholar] [Crossref]

17. Lautrup, A. D., Hyrup, T., Zimek, A., & Schneider-Kamp, P. (2024). SynthEval: A framework for detailed utility and privacy evaluation of tabular synthetic data. Data Mining and Knowledge Discovery, 39(1). https://doi.org/10.1007/s10618-024-01081-4 [Google Scholar] [Crossref]

18. Hernandez, M., Osorio-Marulanda, P. A., Catalina, M., Loinaz, L., Epelde, G., & Aginako, N. (2025). Comprehensive evaluation framework for synthetic tabular data in health: Fidelity, utility and privacy analysis of generative models with and without privacy guarantees. Frontiers in Digital Health, 7, 1576290. https://doi.org/10.3389/fdgth.2025.1576290 [Google Scholar] [Crossref]

19. Wang, S., Maddi, A., Lin, Z., & Fanti, G. (2026). SynAE: A framework for measuring the quality of synthetic data for tool-calling agent evaluations. arXiv:2605.22564. [Google Scholar] [Crossref]

20. Wang, S., Maddi, A., Lin, Z., & Fanti, G. (2026). ESDAE: Evaluating synthetic data for agent evaluation. ICLR 2026 Workshop on Data-Centric Foundation Models. [Google Scholar] [Crossref]

21. Yuan, Y., Cheng, L., Atwal, T., Shi, Z., Tieu, C. N., & Liu, Y. (2026). SynEval: A multidimensional framework for evaluating synthetic data on fidelity, utility, diversity, and privacy. IEEE Symposium on Security and Privacy Poster Proceedings. [Google Scholar] [Crossref]

22. Abdel-Azim, A., Wang, R., & Lin, X. (2026). Harnessing synthetic data from generative AI for statistical inference. Statistical Science (submitted manuscript), arXiv:2603.05396 [Google Scholar] [Crossref]

• From Transcripts to Competencies: A Digital CLER Framework for Malaysian Graduates