Operational Anomaly Detection in Flight Data Using a Multivariate Gaussian Mixture Model

This paper presents a robust real-time aircraft health monitoring framework using a machine learning based approach, specifically the multivariate Gaussian mixture model (mGMM), for the detection of in-air operational anomalies of an aircraft system. Sensor fusion and noise filtering algorithms have also been adopted to reduce dimensionality of the feature space while avoiding the elimination of useful information from the original flight data. Random noise in each feature, induced by the aircraft sensors and data acquisition system, is filtered out using a weighted averaging window while maintaining inherent variances. The filtered dataset is then fused according to the underlying physics of each sensed feature to reduce redundant features and subsequently trained using the mGMM. The methodology allows monitoring the behavior of each feature as well as correlations between features, significantly improving detection sensitivity. The high computational efficiency of this approach permits real-time monitoring of an aircraft system.


INTRODUCTION
There is an urgent need to develop real-time automated system health management (SHM) frameworks to provide accurate assessment of aviation safety for both current and future aircraft systems.Current aircraft health management systems rely on pre-defined subsystem thresholds and binary exceedance criterions to identify operational anomalies which may not be sufficient and accurate in reflecting the current health status of the aircraft.Inflight malfunction of aircraft subsystems may lead to lowered flight performances and unexpected control issues.In order to maintain high standards of operational safety, a high-performance real-time monitoring system must be capable of not only detecting the state malfunctions but also providing suggestions for maintenance to reduce unnecessary downtime of the aircraft (Dalton, Cawley, & Lowe, 2001).However, the development of an SHM framework for such complex system is challenging due to the dynamic interactions of the subsystems and systems, which require simultaneous individual and networked monitoring and analysis to determine global system performance.
Many studies have been conducted to develop fault detection methodologies to ensure system reliability in the presence of subsystem anomalies (Oonk, Maldonado, Figueroa, & Lin, 2012); it should be noted that an anomaly is defined as behavior that shows a significant deviation from standard system behavior.Model-based fault detection techniques are Guoyi Li et al.This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
the most widely used approaches; these are generally classified into quantitative and qualitative approaches (Venkatasubramanian, Rengaswamy, & Kavuri, 2003;Venkatasubramanian, Rengaswamy, Yin, & Kavuri, 2003).The quantitative approach explicitly defines the input-output relation through state-space models such as Kalman filters and observer/parameter estimation-based algorithms (Foo, Zhang, & Vilathgamuwa, 2013;P. M. Frank, 1997;Paul M. Frank, 1994;Zhao, Liu, & Li, 2017).For example, Gilbert et al. (2013) developed extended Kalman filters (EKF) methods for sensor fault detection in synchronous motor driver applications by estimating the phase currents and rotor speed of the motor simultaneously.Zhao et al. (2017) introduced observer based dynamic algorithms for fault tolerant control (FTC) of a nonlinear system with actuator failures based on adaptive dynamic programming (ADP).The qualitative approach utilizes a set of if-then-else rules and their corresponding inferences that find the consequence based on given knowledge, such as digraphs, fault trees and qualitative physics (Bartlett, Hurdle, & Kelly, 2009;Sihombing & Torbol, 2018).Sihombing and Torbol (2018) proposed a parallel fault tree algorithm with a graphical processor unit (GPU) computing scheme, providing increased reliability and effective identification of failures.
In addition to model-based algorithms, data driven approaches have recently been attracting increased attention due to the rapid development of computational power and big data analysis techniques; such approaches typically leverage sufficient amount of historical information containing system features to diagnose and predict the health status.Feature extraction and information fusion techniques have been extensively developed to address the computation efficiency issue associated with the high dimensionality dataset for the data-driven methods (Venkatasubramanian et al., 2003); some examples are principal component analysis (PCA), support vector machine (SVM), and artificial neural networks (ANN) (Banerjee & Das, 2012;Sadough Vanini, Khorasani, & Meskin, 2014;Samanta & Al-Balushi, 2003;Zhang, Sato, & Iai, 2006;Zhou, Zhao, & Cao, 2014).Samanta and Al-Balshi (2003) developed a neural network approach to address the problem of fault diagnostics of rotating bearing systems using time-domain vibrational signals in real-time.Zhang et al (2006) developed a SHM framework for rapid state estimation of large-scale structures by employing incremental SVM based regression data-driven models.
The success of data driven approaches for fault detection has also motivated the development of an SHM framework for aircraft safety, such as the Flight Operations Quality Assurance (FOQA) program designed by the FAA (2004).In this program, exceedance analysis was used to identify the fault conditions based on the state of each flight parameter, such as engine fan speed, control surface position, engine power plant performance, etc.For detailed investigation of discrete flight parameters, longest common subsequence (LCS) and sequence clustering techniques were developed (S.Budalakoti, Srivastava, & Akella, 2006;Suratna Budalakoti, Srivastava, & Otey, 2009); such discrete flight parameters, e.g.flip position, shows fast estimation due to its beneficial characteristics of sequential data.The outlier detection algorithm represents another class of fault detection method that shows promising effectiveness and efficiency.For example, the distance based (DB) outlier algorithm was employed to detect anomaly conditions based on the investigation of the extreme values that deviate from observations in the training examples using clustering methods such as k-nearest neighbor (Bay & Schwabacher, 2003;Knorr, Ng, & Tucakov, 2000).The major advantages of the DB outlier algorithm are that no explicit distribution is required to establish abnormal conditions which significantly reduces computational cost, and no strict limit on feature dimensionality is applied.For further improving the detection accuracy, kernel functions are typically used for feature space transformation to obtain a better representation of the monitored system.Multiple kernel anomaly detection (MKAD) algorithms (Das, Oza, Matthews, & Srivastava, 2007), where a combination of multiple kernel functions are used to construct the feature space, are shown to be suitable for monitoring complex systems.Lishuai Li et al. (2015) suggested clustering-based anomaly detection method, known as clusterAD-flight, to automatically detect faulty conditions based on routine airline flights, outperforming the MKAD approach.Schwabacher, Oza, and Matthews (2009) summarized widely used unsupervised anomaly detection algorithms and demonstrated their performance under different anomaly conditions.
Although many SHM techniques for fault detection have been developed, there is still a need for a fully integrated framework for safety monitoring of an in-air aircraft system; such a framework is expected to explicitly monitor the performance of each sub-system and accurately estimate performance of the complete aircraft system using appropriate metrics.Furthermore, sufficient computational efficiency is required for fulfilling the demand of on-board real-time monitoring and provide safety guidelines under faulty conditions at early stages.Motivated by these issues, this paper presents a robust real-time in-air SHM framework for the detection of operational anomalies using a multivariate Gaussian mixture model (mGMM).Before training the model, random noises contained in sensing signal are filtered using a weighted averaging window, and the redundant features are combined to eliminate dependency which is a requirement for the use of the mGMM model.The methodology is validated using flight data captured through a commercial flight simulator with pre-defined faults.Implementing the developed framework on available airline flight datasets, outliers in the real airline flight data are also flagged and investigated.

Historical Data
In this research, the analysis is performed using Flight data recorder (FDR) information containing on-board aircraft sensor data from commercial flights of the four-engined Boeing 747-400 which is publicly available at the National Aeronautics and Space Administration (NASA) DASHlink network.The chosen data corresponds to 186 flight parameters that are categorized into two classes, based on their features, discrete and continuous.The discrete flight parameters are integers denoting either an on-off state such as landing gear up/down, or a flight status including takingoff, cruising, etc.; the continuous flight parameters include body longitudinal acceleration, position of rudder in degrees, aileron degree and elevator degree, and engine exhaust gas temperature, etc. that are obtained by the on-board sensors and can represent the performance of the aircraft.Therefore, these flight parameters are used to monitor the current condition of aircraft system in this research.Random noise induced by aircraft sensors and the data acquisition system in each feature is filtered through a weighted averaging window while maintaining inherent variances.A robust version of local regression weight linear least square (LOESS) method, available in MATLAB, is used and the window size is set to be 0.2% of the total number of data points.The redundant features are fused to further improve the accuracy and efficiency.For example, the exhausted gas temperature values are averaged into a signal value, since they come from four engines separately, which are identical under most of conditions.

Live Data Generation
Although a massive dataset from the FDR of passenger aircrafts is available, the majority of this information pertains to a healthy flight condition.To perform fault diagnosis however, datasets relating to fault conditions must be present to train and validate the diagnosis algorithms.Such datasets might be obtained from real-world aircraft fault case reports such as the National Transportation Safety Board (NTSB) reports which cover civil transportation accidents including aviation incidents.However, these reports are primarily text based and the avionics data is rarely released or detailed.
Alternatively, these datasets may be simulated using flight simulators that have the ability to simulate realistic flight scenarios with specified faults.In this work, the commercial desktop flight simulator X-Plane-11 is used to simulate the four-engine Boeing 747-400, which is a wide-body commercial passenger jet airliner, at a constant altitude cruising flight phase with realistic weather conditions and faults.The simulation conditions are representative of a real flight phase.Once the aircraft is cruising at the desired flight level various autopilot controls maintain the aircraft speed, heading, and altitude that are programmed using the in-plane flight management computer (FMC).To obtain the training datasets, flight parameters from the simulator is written to a comma separated value (CSV) file during the cruising flight phase.For real-time validation of the fault diagnosis algorithm, the flight parameters from the simulator are streamed in real time into MATLAB using a user datagram protocol (UDP) connection that allows communication between the two applications.The backend machine learning algorithm then processes the data in MATLAB and diagnoses faults in real-time.With this approach, faults can be detected as soon as they occur and the delay between fault occurrence and detection is used as an efficiency metric for the algorithm.Additionally, only a subset of the available flight parameters in X-Plane are considered since the simulated data of the rest of the flight parameters vary substantially from observed real datasets.Only 20 flight parameters, which are the same as those obtained from NASA DASHlink network with a deviation of 5% from the equivalent real datasets, are considered for training and validation purposes.

Learning and Anomaly Detection Algorithm
Accuracy and efficiency are both critical in a real-time in-air fault detection system.The proposed fault detector should be able to provide an early alarm to the pilot and control tower in the event of an anomaly, even if the standard operational threshold(s) of certain aircraft part(s) provided by their vendors are maintained.Second, the dimensionality of the recorded aircraft data is massive; comprising over 150 different features from sensors at multiple locations.Hence, it is a challenge to maintain computational efficiency in both training and detection phases for real-time analysis.Considering these issues, an mGMM has been developed as a real-time fault detector for the aircraft system considered in this work.Gaussian mixture based methodologies have been proven to be accurate and efficient for detecting anomalies in many applications, such as batch process (Chen & Zhang, 2010), semiconductor manufacturing (Yu, 2011), and induction machines (Lemos, Caminhas, & Gomide, 2010).The primary advantage of the mGMM over the general Gaussian mixture model is that not only the distribution of every attribute, i.e., sensing feature, but also the correlation between attributes can be interpreted through the training process.The formulations of the Gaussian mixture model and the mGMM are introduced next.
The vector containing all sensing features at the  ℎ synchronized time step is defined as   () , where  = (1) Importing the Gaussian distribution function, it can be expressed as (2) In order to consider the correlation between attributes, mGMM implements the covariance matrix  into Eq.( 2) instead of multiplication of standard deviations among attributes.Hence, the global safety density estimation function for an mGMM can be expressed as where and where  = 1, 2, … , , and  is the number of samples.To maintain computational efficiency, the mGMM is formulated in a vectorized computation scheme.As a result, the notations in bold in Eq. ( 3) to ( 5) are the vectors that contain all the features with dimension  from the aircraft.For clarification, ,  and  have the dimension of 1× , 1×  and  × , respectively.Comparing mGMM model with GMM model, mGMM is a generalized formulation of GMM, and Gaussian mixture model is a specific case of mGMM, whose covariance matrix just contains non-zero values at diagonal, i.e.,   2 .The vector containing the labels of all samples is denoted .For each sample,   is defined as a binary variable as follows.
where 1 and -1 represent normal flight conditions and anomalies, respectively.The threshold  is defined using a cross-validation process by optimizing the F-1 score that is expressed as where  and  are the precision and recall of developed model, respectively, and can be expressed as and The true positive represents successful prediction of an actual fault case; the number of predicted positive is the total number of cases that are predicted as faults; the actual positive is the number of cases representing real faults.

Real-time Monitoring using Simulated Dataset
The parameters in the cruising phase are extracted as mentioned earlier.For a direct comparison with the real dataset, 20 features including engine core speed, fan speed, exhaust gas temperature, engine oil temperature, etc., are selected as representatives of aircraft performance, and the altitude is maintained within the range of 29,000 to 30,000 feet in the simulations presented here.The results from the developed model with the simulated aircraft anomalies are investigated next.The aircraft states are defined using binary values, "1" indicates the healthy state (normal operation state), which is defined as the current performance of the monitored aircraft, and "-1" represents an anomaly (consistent with Eq. ( 6)).The alarm delays, which is defined as the time difference between the occurrence and the detection of the fault, are also assessed.The oil temperature of the aircraft during the simulated oil temperature faults, which leads to a sudden increase in oil temperature, is presented in Figure 1

Operational Anomaly Detection using Airline Dataset
A total of 458 flights from the FDR datasets are investigated in this study.The parameters in the cruising phase are extracted; same parameters and altitude as mentioned in Section 3.1 are selected for a direct comparison with the simulated dataset; this results in a sample size of 732,000.Due to the high dimensionality of the dataset, the scale of the global performance probability value is very small.Therefore, the distribution of global performance probability is investigated in a logarithmic scale for enhanced visualization, as shown in Figure 3.It should be noted that the F-1 score based cross-validation method is not possible to be used to find the threshold of this dataset, since the status of health of the dataset is not labeled.Therefore, to investigate the cases that possess the largest deviation compared with the normal operations, a threshold of -200 is selected, and the distribution of samples that fall out of this margin are illustrated in Figure 3 in a zoom-in plot.The sampling points whose probabilities are less than -200 in logarithmic scale are regarded as the moments when the aircraft are under abnormal operations; such sample points are found to be in three out of 458 flights.To understand the cause of the detected anomalies, the aircraft dynamic behavior associate with these three flights are investigated.Interestingly, the cruising phases of these three cases show the same tendencies.To illustrate the behavior of these flights, the angle of attack, pitch angle, body longitudinal acceleration, flight path acceleration, and power lever angle of one of the flights with anomalies are investigated.The parameters and their relationships are shown in Figure 4 and the results are presented in Figure 5 to Figure 9.It is found that the angle of attack has a sudden leap, marked in red, and the angle between the longitudinal and horizontal planes of the aircraft body increases, as shown in Figure 5 and Figure 6.Consequently, the body longitudinal acceleration and flight path acceleration experience sudden drops as shown in Figure 7 and 8.It is also observed that the pilot, probably due to the encounter of turbulence, reduces the power lever angle, as shown in Figure 9, which might be the reason for reduction in acceleration and increase in angle of attack.

CONCLUSION
A robust real-time in-air health monitoring framework, using a multivariate Gaussian mixture model (mGMM), has been developed for the detection of operational anomalies.Sensor fusion and noise filtering methodologies have also been utilized to reduce dimensionality and avoid eliminating useful information of flight dataset.A cross-validation model has been used to effectively identify the threshold of global performance of aircrafts.In order to assess the capability of real-time monitoring, a virtual platform has been developed by combining X-Plane flight simulator and UDP connection with MATLAB.The results showed high accuracy in detecting the pre-defined faults with very short alarm delays.The accuracy of this framework has also been demonstrated using available airline FDR datasets from NASA's DASHlink network; the results showed robustness in detection of the operational anomalies in cruising phase.The methodology developed is computationally efficient, which will enable real-time monitoring of aerospace vehicles.
Additionally, both behavior of each feature and correlations between features can be monitored, which significantly improves detection sensitivity.Future work will focus on the operational anomaly detection in other flight phases such as take-off and landing.
For training and F1 score based validation, 7000 samples are simulated as normal operations, while 140 samples are simulated as anomalies that include: a) electrical failure of a full authority digital engine (or electronics) control (FADEC); b) oil pump fault; c) engine stall fault; d) engine driven hydraulic pump fault; e) slow depressurization; f) generator fault; g) engine fire.After training the mGMM model and obtaining the threshold from the cross-validation process, the test set with two separate anomaly cases are investigated: oil pump and compressor stall faults.With the developed live data streaming platform, the trained model monitors the aircraft performance while the flight simulation is running.(a) Oil temperature and (b) health state monitoring of aircraft under oil temperature fault.(a) (b) Figure 2. (a) True airspeed and (b) health state monitoring of aircraft under compressor stall fault.
(a).As shown in Figure1(b), such a fault is detected by the developed model with a three-second alarm delay.Additionally, the compressor stall fault is shown in Figure2(a), which introduces a drop in true airspeed.This fault is also successfully detected, as shown in Figure2(b), with a one-second alarm delay.

Figure 3 .
Figure 3. Logarithmic scale distribution of global performance probability with a zoom-in view of the region less than -200.

Figure 4 .
Figure 4. Demonstration of flight parameters that are described in Figure 5 to Figure 9.

Figure 5 .
Figure 5. Angle of attack under anomalies.