Vehicle State Monitoring and Fault Detection System for Unmanned Ground Vehicles (UGV) using Markov Models

This research presents a novel fault detection and diagnostics system for unmanned ground vehicles (UGVs) by combining Markov models representing the vehicle's navigation, kinematic behavior, and vehicle dynamics systems. Existing studies do not specifically address the challenges related to UGVs and their complex subsystems or the incorporation of weather and environmental condition data. The proposed system leverages environmental and weather condition data to monitor the UGV's state and detect anomalies in its behavior. By predicting the probability of faults such as collisions, sensor damage, and other malfunctions, the system aims to enhance the safety, reliability, and performance of UGVs. The research will demonstrate the effectiveness of the proposed methodology through case studies and performance evaluation, highlighting its potential application in various real-world scenarios. This work contributes to the ongoing research in prognostics and health management, particularly for autonomous systems, by providing a new approach to fault detection and diagnostics in UGVs.


PROBLEM STATEMENT
Unmanned ground vehicles (UGVs) have gained significant attention in recent years due to their diverse applications in transportation, logistics, and hazardous environment exploration.Ensuring the safety and reliability of these vehicles is of paramount importance, as faults or anomalies in their behavior can lead to critical failures, endangering human life and property.Traditional fault detection and diagnostics methods may not be sufficient to address the complex nature of UGV systems, particularly when considering the interdependence of the vehicle dynamics, perception, navigation, and control subsystems, as well as the influence of environmental factors Several studies have been conducted in the field of fault detection and diagnostics for autonomous systems, focusing on various aspects of using Markov models.Especially Hidden Markov Models (HMMs) have been extensively studied in various contexts.Martino (2020) provides an indepth analysis of HMMs for multivariate, functional, and complex data.Similarly, Yu (2010) proposes HMMs that combine local and global information for nonlinear and multimodal process monitoring.Azzalini (2022) employs HMMs as well as Deep Learning (DL) for the detection of anomalies in the behavior of autonomous robots.However, these studies do not specifically address the unique challenges associated with the integration of navigation and kinematic/dynamic subsystems, which is one of the critical aspects of our research.Khreich, Granger, Sabourin, and Miri (2009) focus on combining HMMs for improved anomaly detection, which is directly relevant to our research.Their approach to combining HMMs, offers valuable insights into the integration of Markov models representing the UGV's subsystems.There is a need for a comprehensive study that specifically addresses the challenges of UGV fault detection and diagnostics by integrating Markov models representing navigation, kinematic / dynamic subsystems, and weather/environmental condition data.
The objective of this research is to develop a comprehensive vehicle state monitoring system for UGVs that can effectively detect and diagnose faults by integrating Markov models of the navigation and kinematic/dynamic subsystems, along with information about the weather and environmental conditions.This approach aims to provide a more accurate and robust fault detection system, capable of predicting the probability of faults such as collisions, sensor damage, and other malfunctions, ultimately enhancing the safety and reliability of UGVs.

EXPECTED CONTRIBUTIONS
The outcomes of this research will provide: 1) The development and implementation of a novel vehicle state monitoring system for UGVs that effectively integrates the navigation and kinematic/dynamic subsystems.2) A systematic approach to incorporate weather and environmental condition data into the vehicle state monitoring system.3) Demonstration of the system's effectiveness in detecting anomalies and predicting fault probabilities through case studies and performance evaluation, emphasizing its applicability in a variety of realworld scenarios.4) Comparative analysis of the developed system with existing fault detection and diagnostics methods for UGVs, highlighting its potential to enhance the safety and reliability of these vehicles across diverse applications.These contributions will significantly impact the field of prognostics and health management, particularly for autonomous systems, by offering a new approach to fault detection and diagnostics in UGVs that aims to improve their safety, reliability, and overall performance.

RESEARCH PLAN
The research plan for developing the proposed vehicle state monitoring system for UGVs consists of the following objectives:  Design Markov models representing the navigation and kinematic/dynamic subsystems of the UGV, integrating the state space and mathematical model.
 Develop a detailed mathematical model to calculate the probabilities of being in different states, considering the UGV's position, orientation, vertices and edges, vehicle dynamics, and weather condition data.
 Implement the vehicle state monitoring system by employing a simulation environment to test and evaluate its performance in detecting anomalies and predicting fault probabilities.
 Conduct case studies and performance evaluation of the proposed system, comparing its performance with existing fault detection and diagnostics methods for UGVs.
 Analyze the results obtained from the case studies and identify the advantages and potential improvements offered by the system.
By following this research plan, we aim to provide a comprehensive and novel approach to fault detection and diagnostics in UGVs, contributing to the ongoing research in prognostics and health management for autonomous systems.

Sampling-Based Path Planning
Sampling-based planning has emerged as a vital approach in robotic motion planning, addressing the complexities and computational challenges associated with high-dimensional configuration spaces.Karaman and Frazzoli (2011) argue that the most influential sampling-based motion planning algorithms to date include Probabilistic Roadmap Method (PRM) and Rapidly exploring Random Trees (RRTs).The primary aim of these algorithms is to efficiently explore the configuration space, generate collision-free paths, and ensure convergence to optimal solutions.Sampling-based planning techniques rely on the generation of random samples in the configuration space and the establishment of connections between them to create a roadmap or tree structure.
PRMs, introduced by Kavraki, Svestka, Latombe, and Overmars (1996) are one of the earliest sampling-based planning methods.The PRM algorithm consists of two phases: the roadmap construction phase, which involves sampling and connecting nodes, and the query phase, which finds a path between the start and goal configurations.RRTs, proposed by LaValle (1998), are another fundamental sampling-based planning technique.The RRT algorithm incrementally constructs a tree rooted at the initial configuration by randomly sampling the configuration space and connecting the samples to the nearest vertex in the tree.
Figure 1 shows examples of the path planning graphs generated by the RRT and PRM algorithms for nonholonomic and holonomic robots respectively.Despite the numerous hybrid implementations of samplingbased planning algorithms, the underlying concept of constructing a graph to represent the environment and the obstacles within it remains consistent.In these graphs, nodes represent distinct configurations or states of the robotic system, which could be positions in the workspace, joint angles, or other parameters that define the system's state.Edges, on the other hand, represent feasible transitions between these configurations that avoid collisions with obstacles.By creating such a graph, these algorithms encapsulate the complex relationships between the robot, the environment, and the obstacles, providing a structured and systematic means to explore the configuration space.

Markov Model
To monitor the vehicle's operations, we developed a Markov model based on its actions while traversing the environment.
The Markov property states that the future state of a system depends only on its current state, and not on the sequence of previous states.The transition probabilities for this UGV can be updated based on the vehicle's current position and orientation, and the vertices and edges of the graph.

Calculation of State Probabilities
The following mathematical formulation aims to calculate the probabilities of an unmanned ground vehicle (UGV) being in one of four states: "going straight," "going left," "going right," or "collision."This is done considering the UGV's position and orientation, the vertices and edges of the graph representing the environment.From the graphs in Figure 1 we can see that from any given vertex, the incident edges for that vertex represent the directions the UGV can go in without collision.All the other directions are prone to collision.We divide the vehicle's field of vison in "collision", and "collision-free" areas.Let the vehicle be at position "p" and "u" be the UGV's orientation vector.The field of vision can be represented as a semi-circle with radius equal to the length of the longest edge and center p. V is the set of vertices connected to p.

𝑉 = {𝑣𝑒𝑟𝑡𝑒𝑥 1, 𝑣𝑒𝑟𝑡𝑒𝑥 2, 𝑣𝑒𝑟𝑡𝑒𝑥 3, … 𝑣𝑒𝑟𝑡𝑒𝑥 𝑁}
The "collision free" area can be calculated by assigning a small sector for each edge or neighboring vertex.The unit vector vi representing the direction of the i th edge is given by: Figure 2. Field of Vision for the UGV Each vertex sector can now be classified as a "going straight", "going left", or a "going right" sector based on the angle formed by its direction vector vi, and the UGV's orientation vector u.This angle θ can be calculated by:  = cos  . ‖‖‖ ‖ Each vertex sector is classified into "going straight," "going left," or "going right" based on their angular position relative to the UGV's orientation, θ.Let ε be the threshold angle for classifying an edge as "going straight", based on the UGV's vehicle dynamics.
The angle of the sector, α is the same for each sector.The "collision" and "collision-free" probabilities can be calculated as follows: 'N' is the total number of vertices connected to p.The "collision-free" probability can be divided into the "going straight," "going left," and "going right" probabilities as follows: The resulting probabilities can be used to predict the UGV's state and ultimately, it's behavior in the future enabling fault detection and diagnostics.This mathematical formulation is a key component of the comprehensive vehicle state monitoring system for UGVs proposed in this research.

Fault Diagnosis Process
The proposed fault diagnosis process in this research makes use of the Markov model and the vehicle state monitoring system to diagnose faults in UGVs.This assumes that any fault in the system will result in deviations from the expected state probabilities as described by the Markov model.The Markov model calculates the probabilities of the UGV being in different states, thus generating an expected state sequence under normal conditions.Upon obtaining the expected state sequence, the actual behavior of the UGV is monitored and compared with the predicted behavior.The comparison involves assessing the actual state sequence of the UGV against the expected state sequence.This comparison is facilitated by the vehicle state monitoring system, which continuously tracks the UGV's state transitions.When a discrepancy is detected between the actual and expected state sequences, it signifies an anomaly that could potentially indicate a fault.The nature and magnitude of the discrepancy can provide valuable information about the type and severity of the fault, thus aiding in the diagnosis process.Once a fault is detected and identified, it is then categorized according to its severity and impact on the UGV's performance.This enables the system to prioritize its response according to the severity of the fault.

Remaining Work
The remaining work includes the incorporation of vehicle dynamics, and environmental sensor data to accurately predict the UGV's expected behavior.This will significantly improve the fault detection capability of the vehicle monitoring system.The dynamic states of the vehicle include velocity, acceleration, brake, steering angle, and traction.Weather data can be incorporated into the model as an external factor that influences the transition probabilities between states.Weather conditions can have a significant impact on the behavior of a UGV.For instance, rain, snow, or fog can affect the vehicle's traction, visibility, and sensor performance.One challenge here is that the state space becomes more complex, and so computational techniques may be needed to handle this increased complexity.
Techniques like Hidden Markov Models (HMM), Model Predictive Control (MPC) and Reinforcement learning (RL) can be used to generate an expected behavior of the UGV.By comparing the expected behavior with the actual behavior, anomalies or faults can be detected.Further, the robust or stochastic variants of MPC can account for uncertainties and disturbances, which are common in real-world scenarios.
MPC could be used to model the system's dynamics and predict its behavior under normal conditions, while RL could be used to learn optimal policies to detect anomalies that may indicate faults and responding to them.This could result in a robust and effective fault detection system for UGVs.

CONCLUSION
In this research a novel fault detection and diagnostics system for unmanned ground vehicles (UGVs) is proposed that combines Markov models representing the vehicle's navigation, kinematic behavior, and vehicle dynamics systems.We show that based on the information from the navigational module, collision risk as well as it's expected kinematic behavior can be modeled for a simple UGV.
Expanding our method to include other more complex aspects of an UGV is the immediate next step.Traditional fault detection and diagnostics methods are not sufficient to address the complex nature of UGV systems, considering the interdependence of the vehicle dynamics, perception, navigation, and control subsystems.The proposed approach provides a more accurate and robust fault detection system, capable of predicting the probability of faults such as collisions, sensor damage, and other malfunctions, ultimately enhancing the safety and reliability of UGVs.

Figure 1 .
Figure 1.Path planning graphs generated byRRT (left)   and PRM (right) algorithms.The limitations of pure PRMs and RRTs have led to the development of hybrid and multi-query approaches that combine the strengths of both methods.Algorithms like PRM-RRT, BIT*, SST, and FMT* have popularly found applications in diverse domains such as mobile robotics, aerial robotics, manipulation, and autonomous vehicles.Despite the numerous hybrid implementations of samplingbased planning algorithms, the underlying concept of constructing a graph to represent the environment and the obstacles within it remains consistent.In these graphs, nodes represent distinct configurations or states of the robotic system, which could be positions in the workspace, joint angles, or other parameters that define the system's state.Edges, on the other hand, represent feasible transitions between these configurations that avoid collisions with obstacles.By creating such a graph, these algorithms The state / state transition probabilities of the Markov model can be dynamically calculated based on the path planning graph, position, and orientation of the UGV.Consider a simple UGV with State space:  = {going straight, going left, going right, collision} Transition probability matrix: (straight → straight) ⋯ (straight → collision) ⋮ ⋱ ⋮ (straight → collision) ⋯ (collision → collision)

Figure 2
Figure 2 represents the field of vision of the UGV positioned at p and orientation u.The small circles represent the neighboring vertices, and their individual sectors are colored