Optimizing Deep Reinforcement Learning and Computer Vision for Drone Navigation

The fast development of autonomous aerial systems has given the focus on the necessity of intelligent navigation methods that would be able to operate in complex, dynamic, and unstructured environments. This paper is dedicated to the optimization of Deep Reinforcement Learning (DRL) with the usage of Computer Vision (CV) in autonomous drone navigation with the focus on the simulation-based testing of the model performance before its practical application.. The study has followed a simulation-based approach with a combination of Proximal Policy Optimization (PPO) of reinforcement learning and computer vision models of convolutional neural networks (ResNet50 and YOLOv5). The simulation environment is a recreation of diverse scenarios such as indoor, urban, forested and open field scenarios. It has 500 flight episodes, in which the performance of the UAV is measured with the key metrics including rate of reaching target, number of collisions, total rewards, navigation accuracy, convergence rate of DRL algorithms, and power consumption. The most important results show that the integrated DRA-CV model reached a target-reaching success rate of 96 percent and also had an average number of collisions in the form of 1.33 per episode. PPO algorithm performed better than DQN, A3C and SAC in terms of convergence, and optimal policies were obtained on an average of 177.5 episodes. CNN-based visual perception was able to identify obstacles with 94 percent accuracy of the obstacle with a low rate of false positive (3 percent) and false negative (2 percent) to navigate a dynamic environment safely. The average cumulative reward was 1847 units and the energy consumption was optimised to 1184.7 Joules which proved to be an efficient use of resources..

Deep Reinforcement Learning, Computer Vision, UAV Navigation, Autonomous Drones, Simulation, Obstacle Avoidance, Path Planning

1. Chen, C., Sun, H., & Zhao, L. (2020). Deep reinforcement learning for vision-based UAV navigation in GPS-denied environments. Robotics, 9(4), 87. https://www.mdpi.com/2218-6581/9/4/87 [Google Scholar] [Crossref]

2. Han, Z., Liu, W., & Zhou, J. (2022). DRL-based visual navigation for UAVs in dynamic urban scenarios. Robotics and Autonomous Systems, 153, 104071. [Google Scholar] [Crossref]

3. https://www.sciencedirect.com/science/article/pii/S092188902200030X [Google Scholar] [Crossref]

4. Luo, T., & Tang, Y. (2021). DRL for UAV autonomous navigation with real-time obstacle avoidance. Sensors, 21(15), 5090. https://www.mdpi.com/1424-8220/21/15/5090 [Google Scholar] [Crossref]

5. Mao, Y., Chen, J., & Li, P. (2021). End-to-end DRL for autonomous drone navigation using monocular images. Robotics and Autonomous Systems, 141, 103742. [Google Scholar] [Crossref]

6. https://www.sciencedirect.com/science/article/pii/S0921889021001125 [Google Scholar] [Crossref]

7. Mehta, P., & Hwang, J. (2021). Safe reinforcement learning for drone navigation in cluttered environments. IEEE Robotics and Automation Letters, 6(2), 2637–2644. [Google Scholar] [Crossref]

8. https://ieeexplore.ieee.org/document/9381743 [Google Scholar] [Crossref]

9. Park, J., & Kim, H. (2021). Vision-guided deep reinforcement learning for quadrotor navigation in cluttered environments. Sensors, 21(20), 6842. https://www.mdpi.com/1424-8220/21/20/6842 [Google Scholar] [Crossref]

10. Qu, Y., Zhang, Q., & Liu, Y. (2022). UAV navigation with DRL using depth images. Sensors, 22(3), 976. https://www.mdpi.com/1424-8220/22/3/976 [Google Scholar] [Crossref]

11. Sharma, R., & Kumar, S. (2021). Integration of computer vision and deep reinforcement learning for autonomous drone flight. Journal of Intelligent & Robotic Systems, 102(2), 1–17. [Google Scholar] [Crossref]

12. https://link.springer.com/article/10.1007/s10846-021-01338-w [Google Scholar] [Crossref]

13. Wang, J., Chen, Y., & Zhao, Q. (2021). Hierarchical deep reinforcement learning for autonomous drone navigation. IEEE Transactions on Cybernetics, 51(12), 6072–6084. [Google Scholar] [Crossref]

14. https://ieeexplore.ieee.org/document/9444923 [Google Scholar] [Crossref]

15. Yang, L., Zhang, K., Liu, J., & Zhou, D. (2021). Vision-based autonomous drone navigation with deep reinforcement learning. Sensors, 21(5), 1765. https://www.mdpi.com/1424-8220/21/5/1765 [Google Scholar] [Crossref]

16. Yang, Z., Gao, W., & Wang, H. (2022). Deep reinforcement learning with curriculum learning for UAV obstacle avoidance. Robotics and Autonomous Systems, 151, 103949. [Google Scholar] [Crossref]

17. https://www.sciencedirect.com/science/article/pii/S0921889021002125 [Google Scholar] [Crossref]

• What the Desert Fathers Teach Data Scientists: Ancient Ascetic Principles for Ethical Machine-Learning Practice
• Comparative Analysis of Some Machine Learning Algorithms for the Classification of Ransomware
• Comparative Performance Analysis of Some Priority Queue Variants in Dijkstra’s Algorithm
• Transfer Learning in Detecting E-Assessment Malpractice from a Proctored Video Recordings.
• Dual-Modal Detection of Parkinson’s Disease: A Clinical Framework and Deep Learning Approach Using NeuroParkNet