Integration of Condition Information in UAV Swarm Management to increase System Availability in dynamic Environments
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Abstract
The approach of prognostics and health management (PHM) focuses on the real-time health assessment of a system under its actual operating condition and even extending this by the prediction of the future state based on up-to-date system information. This pursues the aim to derive more advanced maintenance or asset deployment strategies in order to keep the operation of the system safe and reliable. In this context, the outcome of a PHM system is often used as a decision support. For a high fidelity system where the actual state is considered at every timestep and a decision is executed immediately based up on this information, Reinforcement Learning (RL) becomes a tool to find an optimized solution. Therefore the paper presents a methodology that integrates health and operational data into a RL approach in order to derive immediate operational strategies for lower degradation and higher safety and reliability. The approach is evaluated on the basis of a swarm of unmanned aerial vehicles (UAVs) that performs a complete-area path-coverage (CAPC) mission. It can be shown that the integration of health information as well as environmental data describing dynamic operating conditions lead to lower degradation and result in more reliable operations of the swarm while achieving a more flexible mission performance compared to pre-divided swarm-missions. Varying states are also taken into account, which emphasises this approach to be a highly dynamic PHM system application.
How to Cite
##plugins.themes.bootstrap3.article.details##
reinforcement learning, reliability, swarm, cooperation, distributed systems, machine learning, artificial intelligence
Andersson, K., Bang, M., Marcus, C., Persson, B., Sturesson, P., Jensen, E., & Hult, G. (2015). Military utility: A proposed concept to support decision-making. Technology in society, 43, 23–32.
Bougacha, O., & Varnier, C. (2020). Enhancing decisions in prognostics and health management framework. International Journal of prognostics and health management, 11(1).
Cho, S.-W., Park, J.-H., Park, H.-J., & Kim, S. (2021). Multiuav coverage path planning based on hexagonal grid decomposition in maritime search and rescue. Mathematics, 10(1), 83.
Darrah, T., Qui˜nones-Grueiro, M., Biswas, G., & Kulkarni, C. S. (2021). Prognostics based decision making for safe and optimal uav operations. In Aiaa scitech 2021 forum (p. 0394).
Hare, J. (2019). Dealing with sparse rewards in reinforcement learning. arXiv preprint arXiv:1910.09281.
Heier, H., Mehringsk¨otter, S., & Preusche, C. (2018). The use of phm for a dynamic reliability assessment. In 2018 ieee aerospace conference (pp. 1–10).
Kim, N.-H., An, D., & Choi, J.-H. (2017). Prognostics and health management of engineering systems. Switzerland: Springer International Publishing.
Kouzehgar, M., Meghjani, M., & Bouffanais, R. (2020).
Multi-agent reinforcement learning for dynamic ocean monitoring by a swarm of buoys. In Global oceans 2020: Singapore–us gulf coast (pp. 1–8).
Liang, E., Liaw, R., Nishihara, R., Moritz, P., Fox, R., Goldberg, K., . . . Stoica, I. (2018). Rllib: Abstractions for distributed reinforcement learning. In International conference on machine learning (pp. 3053–3062). Mahmoud Zadeh, S., Yazdani, A., Elmi, A., Abbasi, A., & Ghanooni, P. (2022). Exploiting a fleet of uavs for monitoring and data acquisition of a distributed sensor network. Neural Computing and Applications, 1–14. Marques, H., & Giacotto, A. (2019). Prescriptive maintenance: Building alternative plans for smart operations. In The 10th aerospace technology congress. Puente-Castro, A., Rivero, D., Pazos, A., & FernandezBlanco, E. (2022). Uav swarm path planning with reinforcement learning for field prospecting. Applied Intelligence, 52(12), 14101–14118.
Radzki, G., Bocewicz, G., Goli´nska-Dawson, P., JasiulewiczKaczmarek, M., Witczak, M., & Banaszak, Z. (2021). Periodic planning of uavs’ fleet mission with the uncertainty of travel parameters. In 2021 ieee international conference on fuzzy systems (fuzz-ieee) (pp. 1–8).
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.