Real-Time Energy Management of PV-ESS Integrated Active Distribution Networks Using Digital Twin-Enabled Deep Reinforcement Learning

The increasing penetration of photovoltaic (PV) generation and battery energy storage systems (BESSs) has significantly increased the operational complexity of active distribution networks, where real-time energy management must simultaneously address renewable uncertainty, voltage regulation, and battery lifetime preservation. Existing Digital Twin-based energy management approaches primarily support monitoring and visualization, whereas deep reinforcement learning (DRL) controllers are commonly developed independently of real-time system synchronization, limiting their adaptability under rapidly changing operating conditions. To overcome these limitations, this paper proposes a Digital Twin-enabled Deep Reinforcement Learning (DT-DRL) framework for coordinated PV–BESS energy management in active distribution networks. The proposed framework establishes a closed-loop cyber–physical architecture in which continuously synchronized Digital Twin states are directly incorporated into a Proximal Policy Optimization (PPO)-based decision-making process. A multi-objective formulation is developed to jointly minimize operating cost, voltage deviation, and battery degradation while satisfying network operational constraints. Renewable generation and load uncertainties are represented using Monte Carlo-based stochastic scenarios to improve policy robustness under practical operating conditions. The proposed framework is validated on the IEEE 33-bus distribution system and compared with rule-based control (RBC), optimal power flow (OPF), and conventional DRL approaches. Simulation results demonstrate that the proposed method reduces the daily operating cost by 22.2%, decreases the maximum voltage deviation to 0.039 p.u., and achieves more stable BESS operation with lower operational variability under uncertain conditions. Furthermore, the complete Digital Twin synchronization and PPO decision-making process requires only 0.41 s per control interval, satisfying the timing requirements of distribution-level energy management systems. These results demonstrate that the proposed DT-DRL framework provides an accurate, computationally efficient, and practically deployable solution for real-time energy management in renewable-rich active distribution networks.

Battery energy storage systems, Deep reinforcement learning, Digital Twin; Energy management, Photovoltaic generation.

1. Liu, Z., Zhang, Y., & Li, H. (2021). Digital twin for power systems: State-of-the-art and future trends. Renewable and Sustainable Energy Reviews, 152, 111675. https://doi.org/10.1016/j.rser.2021.111675 [Google Scholar] [Crossref]

2. Zhou, K., Yang, S., & Shao, Z. (2021). Digital twin-based optimization for energy systems. Applied Energy, 285, 116419. https://doi.org/10.1016/j.apenergy.2020.116419 [Google Scholar] [Crossref]

3. Musa, U. I., & Ghosh, S. (2023). Advancing digital twin through the integration of new AI algorithms. International Research Journal of Engineering and Technology, 922–932. https://www.researchgate.net/publication/369693333 [Google Scholar] [Crossref]

4. Mohamed, N. B., Kaddoum, G., & Hassan, M. Z. (2026). Digital twin-driven continual deep reinforcement learning for coexistence of multiple radio access technology IoT links with nonlinear receivers. IEEE Transactions on Machine Learning in Communications and Networking, 4, 591–611. https://doi.org/10.1109/TMLCN.2026.3669837 [Google Scholar] [Crossref]

5. Kabir, M. R., Halder, D., & Ray, S. (2024). Digital twins for IoT-driven energy systems: A survey. IEEE Access, 12, 177123–177143. https://doi.org/10.1109/ACCESS.2024.3506660 [Google Scholar] [Crossref]

6. Kumari, N., Sharma, A., Tran, B., Chilamkurti, N., & Alahakoon, D. (2023). A comprehensive review of digital twin technology for grid-connected microgrid systems. Energies, 16(14), 5525. https://doi.org/10.3390/en16145525 [Google Scholar] [Crossref]

7. Guerrero, J. M., Vasquez, J. C., Matas, J., Vicuña, L. G., & Castilla, M. (2011). Hierarchical control of droop-controlled AC and DC microgrids—A general approach toward standardization. IEEE Transactions on Industrial Electronics, 58(1), 158–172. https://doi.org/10.1109/TIE.2010.2066534 [Google Scholar] [Crossref]

8. Parisio, A., Rikos, E., & Glielmo, L. (2014). A model predictive control approach to microgrid operation optimization. IEEE Transactions on Control Systems Technology, 22(5), 1813–1827. https://doi.org/10.1109/TCST.2013.2295737 [Google Scholar] [Crossref]

9. Glavic, M., Moreno, R., Bi, T., & Shahidehpour, M. (2021). Reinforcement learning for electric power system decision and control: Past considerations and perspectives. IEEE Transactions on Smart Grid, 12(6), 5128–5142. https://doi.org/10.1109/TSG.2021.3070934 [Google Scholar] [Crossref]

10. Zhang, Z., Wang, J., & Chen, B. (2021). Deep reinforcement learning for energy management in microgrids. Applied Energy, 288, 116618. https://doi.org/10.1016/j.apenergy.2021.116618 [Google Scholar] [Crossref]

11. Ugwu, C. N., Ogenyi, F. C., Ugwu, J. N., Ugwu, P. C. O., & Okon, M. B. (2026). Integration of AI-driven digital twins for real-time optimization of renewable energy grids. Frontiers in Energy Research, 14, 1–13. https://doi.org/10.3389/fenrg.2026.1748233 [Google Scholar] [Crossref]

12. Xu, B., Oudalov, A., Ulbig, A., & Andersson, G. (2018). Modeling of lithium-ion battery degradation for cell life assessment. IEEE Transactions on Smart Grid, 9(2), 1131–1140. https://doi.org/10.1109/TSG.2016.2578950 [Google Scholar] [Crossref]

13. Xu, B., Shi, Y., Kirschen, D. S., & Zhang, B. (2018). Optimal battery participation in frequency regulation markets considering degradation cost. IEEE Transactions on Power Systems, 33(5), 4908–4918. https://doi.org/10.1109/TPWRS.2018.2799437 [Google Scholar] [Crossref]

14. Ton, T. N., Le, P. M., Pham, L. H., & Le, T. M. (2025). Multi-objective optimization of electric distribution systems with integrated distributed generation using deep reinforcement learning. Engineering, Technology & Applied Science Research, 15(2), 22166–22171. [Google Scholar] [Crossref]

• The Indirect Effect of Liquidity and Activity on Company Value with Profitability as an Intervening Variable
• Effect of Financial Skills, Knowledge, and Attitude on The Financial Behaviour of Clergy
• A Decade of Review: Trends in Budget Execution and Financial Performance of Development Projects in Tanzania (2014/15-2023/24)
• The Influence of Pre-Project Planning on the Budget Absorption Rate of Public Funded Infrastructure Projects in Kenya a Comparative Case Study of Narok, Migori, and Kisii County Government Projects
• Assessment of Factors Influencing Digital Transformation in Hotels’ Facility Management in Abuja Metropolis, Nigeria