Real-Time Traffic Signal Optimisation Using Deep Q-Network Algorithm and Camera Data

Traffic congestion has become a problem in developing countries’ urban areas. This is largely caused by traffic signals that have fixed-timing which causes them to fail to adapt to changing traffic conditions in real-time. This research introduces a Reinforcement Learning-based solution using a Deep Q-Network algorithm to optimise traffic signal lights control, aiming at reducing congestion and enhancing traffic flow efficiency. The system is developed in a virtual environment using PTV VISSIM simulation software and the real-time traffic data is collected using simulated cameras. The collected traffic data is then processed using Deep Q-Network algorithm which is implemented using Python and TensorFlow. By optimising traffic signal light timings to be adaptive, the system introduces significant improvements in reducing traffic waiting times at intersections and improving traffic flow on the road in comparison to the traditional fixed-timing systems. The system ensures scalability and effectiveness in offering a promising framework for adaptive traffic management in urban roads.

Traffic Signal Optimisation, Deep Q-Network, Reinforcement Learning, Urban Mobility, Real-Time Simulation, PTV VISSIM.

1. Liu, B., Liu, X., Chen, C., Huang, J., & Ding, Z. (2023). Decentralized Multi-Agent Reinforcement Learning for Traffic Signal Control. In 2023 42nd Chinese Control Conference (CCC) (pp. 6045-6050). IEEE. [Google Scholar] [Crossref]

2. Liu, J., Qin, S., Su, M., Luo, Y., Wang, Y., & Yang, S. (2023). Multiple Intersections Traffic Signal Control Based on Cooperative Multi-agent Reinforcement Learning. Information Sciences, 647, 119484. [Google Scholar] [Crossref]

3. Lu, J., Li, B., Li, H., & Al-Barakani, A. (2021). Expansion of City Scale, Traffic Modes, Traffic Congestion, and Air Pollution. Cities, 108, 102974. [Google Scholar] [Crossref]

4. Kolat, M., Kővári, B., Bécsi, T., & Aradi, S. (2023). Multi-agent Reinforcement Learning for Traffic Signal Control: A Cooperative Approach. Sustainability, 15(4), 3479. [Google Scholar] [Crossref]

5. Qi, F., He, R., Yan, L., Yao, J., Wang, P., & Zhao, X. (2022, August). Traffic Signal Control with Deep Q-Learning Network (DQN) Algorithm at Isolated Intersection. In 2022 34th Chinese Control and Decision Conference (CCDC) (pp. 616-621). IEEE. [Google Scholar] [Crossref]

6. Jiang, S., Huang, Y., Jafari, M., & Jalayer, M. (2021). A Distributed Multi-agent Reinforcement Learning with Graph Decomposition Approach for Large-scale Adaptive Traffic Signal Control. IEEE Transactions on Intelligent Transportation Systems, 23(9), 14689-14701. [Google Scholar] [Crossref]

7. Cornell University, 2019. Diagnosing Reinforcement Learning for Traffic Signal Control. [Online] [Google Scholar] [Crossref]

8. Available at: https://arxiv.org/abs/1905.04716 [Google Scholar] [Crossref]

9. [Accessed 24 September 2024]. [Google Scholar] [Crossref]

10. Ge, H., Gao, D., Sun, L., Hou, Y., Yu, C., Wang, Y., & Tan, G. (2021). Multi-agent Transfer Reinforcement Learning with Multi-view Encoder for Adaptive Traffic Signal Control. IEEE Transactions on Intelligent Transportation Systems, 23(8), 12572-12587. [Google Scholar] [Crossref]

11. Jiang, X., Zhang, J. and Wang, B. (2022) ‘Energy-Efficient Driving for Adaptive Traffic Signal Control Environment via Explainable Reinforcement Learning’, Applied Sciences (Switzerland), 12(11). Available at: https://doi.org/10.3390/app12115380. [Google Scholar] [Crossref]

12. Kim, D. and Jeong, O. (2020) ‘Cooperative Traffic Signal Control with Traffic Flow Prediction in Multi-intersection’, Sensors (Switzerland), 20(1). Available at: https://doi.org/10.3390/s20010137. [Google Scholar] [Crossref]

13. Munuhwa, S., 2020. Approaches for Reducing Urban Traffic Congestion in the City of Harare. [Online] Available at: https://www.researchgate.net/publication/361860503_Approaches_for_Reducing_Urban_Traffic_Congestion_in_the_City_of_Harare [Accessed 11 October 2024]. [Google Scholar] [Crossref]

14. Papageorgiou, M., Diakaki, C., Dinopoulou, V., Kotsialos, A., & Wang, Y. (2003). Review of Road Traffic Control Strategies. Proceedings of the IEEE, 91(12), 2043-2067. [Google Scholar] [Crossref]

15. Ma, J., Li, C., Hong, L., Wei, K., Zhao, S., Jiang, H., & Qu, Y. (2025). Vision-based attention deep q-network with prior-based knowledge. Applied Intelligence, 55(6), 565. [Google Scholar] [Crossref]

16. Zheng, G. et al. (2019) ‘Diagnosing Reinforcement Learning for Traffic Signal Control’. Available at: http://arxiv.org/abs/1905.04716. [Google Scholar] [Crossref]

17. Spatharis, C. and Blekas, K. (2024) ‘Multiagent Reinforcement Learning for Autonomous Driving in Traffic Zones with Unsignalized Intersections’, Journal of Intelligent Transportation Systems: Technology, Planning, and Operations, 28(1), pp. 103–119. Available at: https://doi.org/10.1080/15472450.2022.2109416. [Google Scholar] [Crossref]

18. Wang, T., Cao, J. and Hussain, A. (2021) ‘Adaptive Traffic Signal Control for Large-scale Scenario with Cooperative Group-based Multi-agent Reinforcement Learning’, Transportation Research Part C: Emerging Technologies, 125, p. 103046. Available at: https://doi.org/10.1016/J.TRC.2021.103046. [Google Scholar] [Crossref]

19. Haimerl, M., Colley, M. and Riener, A. (2022) ‘Evaluation of Common External Communication Concepts of Automated Vehicles for People with Intellectual Disabilities’, Proceedings of the ACM on Human-Computer Interaction, 6(MHCI). Available at: https://doi.org/10.1145/3546717. [Google Scholar] [Crossref]

20. Cui, H. et al. (2020) ‘Convolutional neural network for recognizing highway traffic congestion’, Journal of Intelligent Transportation Systems: Technology, Planning, and Operations. Taylor and Francis Inc., pp. 279–289. Available at: https://doi.org/10.1080/15472450.2020.1742121. [Google Scholar] [Crossref]

• What the Desert Fathers Teach Data Scientists: Ancient Ascetic Principles for Ethical Machine-Learning Practice
• Comparative Analysis of Some Machine Learning Algorithms for the Classification of Ransomware
• Comparative Performance Analysis of Some Priority Queue Variants in Dijkstra’s Algorithm
• Transfer Learning in Detecting E-Assessment Malpractice from a Proctored Video Recordings.
• Dual-Modal Detection of Parkinson’s Disease: A Clinical Framework and Deep Learning Approach Using NeuroParkNet