Real time diagnostics and prognostics of UAV lithium-polymer batteries

This paper examines diagnostics and prognostics of Lithium-Polymer (Li-Po) batteries for unmanned aerial vehicles (UAVs). Several discharge voltage histories obtained during actual indoor flights constitute the training data for a data-driven approach, utilizing the Non-Homogenous Hidden Semi Markov model (NHHSMM). NHHSMM is a suitable candidate as it has a rich mathematical structure, which is capable of describing the discharge process of Li-Po batteries and providing diagnostic and prognostic measures. Diagnostics and prognostics in unseen data are obtained and compared with the actual remaining flight time in order to validate the effectiveness of the selected model.


INTRODUCTION
Unmanned aerial vehicles (UAVs) have received great attention during the last decade due to their potential to provide solutions in a wide set of fields and applications, especially in extreme and harsh environments.The remaining flight time of UAVs depends on many parameters such as payload, mission, temperature conditions, wind conditions etc.However, one of the limitations of UAVs is the fact that this dependency is not known.Thus, it is not reliable to use UAVs for long time missions since one unpredicted discharge of the battery during the mission can result to events like loss of equipment and mission failure.As a result, the need for reliable diagnostic and prognostic models, which can estimate in real-time the current state of health (SOH) and the remaining flight time or remaining useful life (RUL), rises and draws increasing attention in the last few years.
The SOH was selected as diagnostic measure since it is one of the critical parameters for indication of the remaining energy and it is regarded as one of the key parameters of battery management system (Ng, Moo, Chen & Hsieh, 2009).However, battery diagnostics cannot be measured directly.In the literature, a number of battery diagnostic approaches have been proposed, such as Ampere-hour integral method (Yang, Zhang & Li, 2015), open-circuit voltage method (Lee, Kim, Lee & Cho, 2008), Kalman filter (Yuan, Wu &Yin, 2013), particle filter (Schwunk, Armbruster, Straub, Kehl & Vetter, 2013), artificial neural networks (Sbarufatti, Corbette, Giglio & Cadini, 2017), radial neural network, support vector machines (Hansen & Wang, 2005) network and wavelet neural network (Gao, Song & Dougal, 2005).Battery models are not necessary to establish and the estimation processes are easy to implement regarding the aforementioned models.However, despite the simplicity, these models cannot meet the demand of precision (Xia, Cui, Sun, Lao, Zhang, Wang, Sun, Lai & Wang, 2018).RUL is defined as the time when equipment performance degrades to the failure threshold for the first time (Escobar, Meeker, Luis, William & Meeker, 2006).If the RUL can be predicted accurately, actions to repair and maintain the equipment can be taken.RUL methodologies can be divided into model-based and data-driven methods (Chen & Pecht, 2012).The discharge process of Lithium-Polymer (Li-Po) batteries is a nonlinear and time-varying dynamic electrochemical process.Therefore, model-based approaches cannot be easily implemented since the discharging process involves a lot of parameters and complex calculations.In consequence, it is not suitable for real-time prognostics.Instead, model-based approaches are used more in theoretical research and battery designation (Lin, Tang & Wang, 2015).
On the other hand, data-driven approaches do not require prior knowledge of the physics of the system since datadriven approaches rely on measured data in order to derive the discharge process of Li-Po batteries.In (Cadini, Sbarufatti, Cancelliere & Giglio, 2019), a method for predicting the end of discharge (EoD) of Li-Ion batteries was recently proposed.This study combined particle filters with an Radial Basis Function Neural Network (RBF-NN).The proposed algorithm was validated successfully on dataset containing measurements of the voltage at the terminals of a Lithium-ion (Li-ion) battery, obtained during charge-discharge laboratory tests at constant current.Si in (Si, 2015) models the degradation process as Wiener process with the dynamic part being described by Brownian motion added to a nonlinear drift function.Kalman filtering was applied to update a key parameter in the drifting function through treating this parameter as an unobserved state variable simultaneously and recursively.The expectation maximization algorithm was used in conjunction with Kalman smoother to achieve this aim.The probability density function of the estimated RUL was derived with an explicit form and data from the NASA AMES Prognostics Center of Excellence Battery Dataset are used for validation.A similar approach was followed in (Dong, Che, Wei & Ling, 2018), a Brownian motion based degradation model was combined with particle filtering.The proposed model tackles the capacity degradation as the traveling distance of a Brownian particle in a given time interval.Then, the PF is used to estimate the drift parameter of the Brownian Motion.The NASA Ames Prognostics Center of Excellence Battery Dataset was used.In (Yan, Zhang, Zhao, Weddington & Niu, 2017), an approach based on Lebesgue Sampling is utilized for Li-Ion batteries RUL estimations.The proposed approach accommodates properly the changing of fault dynamics and provides precise mean RUL estimations including uncertainty quantification.
The contribution of the present study involves the application of a more generic statistical model, that is, the Non-Homogeneous Hidden Semi Markov model (NHHSMM) for the very first time in the topic of diagnostics and prognostics for Li-Po batteries.NHHSMM is the most generic version of Hidden Markov models (HMMs).HMMs were initially introduced and studied in the late 1960s and early 1970s (Rabiner, 1989).However, they became increasingly popular recently.Peng & Dong (2010), highlighted that HMMs have a very rich mathematical structure and can form a solid theoretical foundation for use in engineering applications.An added benefit of employing HMMs is the ease of model interpretation in comparison with pure 'black-box' modeling methods such as artificial neural networks that are often employed in advanced diagnostic/prognostic models (Baruah & Chinnam, 2005).However, HMMs' main disadvantage is that they assume an exponentially distributed state duration (sojourn time) which is not always the case.HSMMs relax this assumption allowing the unconstrained modeling of sojourn time.HSMMs have been utilized successfully for prognostic RUL estimations in condition monitoring of machines ( (Peng & Dong, 2011), (Dong & He, 2007)) for a condition-based maintenance framework.In HMMs and HSMMs, there is the limitation that the state transitions are not dependent on the age of the engineering asset or on the sojourn time in the current state.This is apparently not the case in real-life engineering systems.The work of Moghaddass & Zuo (2014) extended the HSMMs to NHHSMMs.According to NHHSMM, state transitions are a dynamic procedure, which depends on the current hidden state, the time spent in this state (sojourn time), and the total age of the engineering asset.
In the present study, the NHHSMM considers the discharge process in Li-Po UAVs batteries as a stochastic hidden process, which can be estimated from observations (monitoring data) i.e. discharge voltage data of ten actual indoor flights due to the ease of its measurement.The rest of the article is structured as follows.In section 2, a datadriven SOH/RUL methodology is presented.Section 3 summarizes the results of SOH diagnosis and RUL prognosis.The paper closes with the conclusions drawn from this study.

METHODOLOGY
As already mentioned in introduction there is a need for developing data-driven models, which will be capable to describe the discharge process of Li-Po batteries and estimate diagnostic and prognostic measures.Battery diagnostics focus on estimating the SOH while prognostics estimate of the RUL until the end of discharge.Figure 1 summarizes the SOH/RUL prediction methodology, which consists of two parts; the training and testing process.The training process contains the training data and the stochastic model, i.e.NHHSMM, while the testing process uses the training process' output, the extracted testing monitoring data and the diagnostic/prognostic measures.
Figure 1.Flowchart of the SOH/RUL prediction methodology.

NHHSMM
This subsection briefly reviews the fundamentals of the NHHSMM.The interested reader can refer to Moghadass and Zuo (2014) and Eleftheroglou and Loutas (2016) for a more detailed description.The NHHSMM consists of a bidimensional stochastic process.The first process forms a finite Semi Markov chain, which is not directly observed, and the second process, conditioned on the first one, forms a sequence of independent random monitoring data variables.
In order to describe the aforementioned bi-dimensional stochastic process the model's parameters θ have to be estimated.
The parameter estimation consists of the initialization and training procedure.The purpose of the initialization procedure is to identify a set of parameters ζ, with high computational efficiency, which will associate the damage accumulation phenomenon and the available monitoring data.The purpose of the training procedure is to estimate the parameters θ={Γ,Β}.Γ parameters characterize the transition rate distribution between the health (hidden) states, while Β parameters deal with the correlation between the health states and monitoring data.This correlation is represented in a nonparametric and discrete form via a matrix called emission matrix.The purpose of the training procedure is to estimate the parameters θ based on the selected parameters ζ.The complete model M is defined when ζ and θ are known, M={ζ,θ}.
The initialization procedure is obtained by defining the following parameters:  Number of health states (Ν).N refers to the number of discrete levels of discharge.However, health states are not quantitatively but just qualitatively correlated with the discharge Li-Po battery process.The main assumption in this paper is that the battery starts to operate fully charged, health state 1, until the battery voltage dropped to a defined critical level i.e. state N.The final health state N is not hidden but self-announcing and always corresponds to the EoD threshold.
 Transition between the health states (Ω).This parameter defines the connectivity between the N selected health states and it can be soft (gradual transition to neighbour hidden state), hard (sudden transition from any health state to failure state N) and multistep (transition to an intermediate state between the current health state and the failure state).Figure 2 illustrates the three possible types of transition.
 Transition rate function (λ).This parameter is the main describer of the discharge process since each transition is going to follow this λ transition rate function.The transition process can depend on the involved health states (Markovian property), the sojourn time of the current health state, the total operation time (aging) and any other combination between the aforementioned parameters.The most commonly used distributions for the λ function are the Weibull, Gaussian, Exponential and Gamma failure rates.In this study the Weibull failure rate is used since it is the most generic one.
 Discrete monitoring indicator space (Z={z1,z2,…,zV}).The selection of this parameter is crucial for the observation process since emission matrix has N (number of hidden states) rows and V (number of discrete condition monitoring values) columns.The entry in the element (i,j) of the emission matrix represents the probability that zj monitoring value is observed when the system is in health state i.
To summarize, the initialization topology can be denoted as ζ={N,Ω,λ,V}.With regards to the training procedure, parameters θ={Γ,Β} are obtained via the maximum likelihood estimation.Moghadass and Zuo (2014) proposed a method for defining the Maximum Likelihood Estimator (MLE) θ* of the model parameter setting initial values for Γ, Β and solving the aforementioned optimization problem, the parameter estimation process is obtained and diagnostics and prognostics can be estimated.

Diagnostics
Finding a monotonic degradation measure, which demonstrates qualitatively the discharge process of Li-Po battery has always been an interesting and challenging topic in UAVs applications.Therefore, the estimation of the Most Likely State (MLS), which has been proposed by Moghaddass and Zuo (2014), can be used in order to estimate the state of health (SOH) and determined via Eq. (2).

SOH(t|y
This measure maximizes the probability Pr(Q t = i|y 1:t ,  * ) of being at the health state i at the time point t given the monitoring data up to time t.With M*={ζ, θ*} a specific model topology is denoted.

Prognostics
Prognostic measures can be defined based on the θ * parameters and the testing data.In other words, conditional to the testing data and the complete model M * ={ζ, θ * }, prognostics tries to estimate the probability of being in health states 1,…,N-1 at a specific time points in future i.e. the conditional reliability function.Conditional reliability function, R (t|y 1:t p , L > t p ,  * ) = Pr (L > t|y 1:t p , L > t p ,  * ) (3), represents the probability that the battery continues to operate after a time t, less than EOD time L (L>t), further than the current time tp given that the battery has not discharge yet (L>tp), the testing data y1:tp and the complete model M * .In this study the mean and confidence intervals of RUL are proposed as prognostic measures.These measures were calculated via the cumulative distribution function (CDF) of RUL (Moghaddass & Zuo, 2014).The CDF of RUL is defined at any time point via the conditional reliability according to the following equation: Pr (RUL t p ≤ t|y 1:t p ,  * ) = 1 − R(t + t p │y 1:t p ,  * ) (4)

Li-Po data set description
The discharge voltage is used as a health indicator of the Li-Po batteries since the voltage can be easily monitored in real time and most battery powered systems cannot operate once the voltage falls below a critical threshold (EOD).Therefore, the RUL prediction is associated to the time remaining until this threshold is reached.The experimental data consist of 10 battery discharge histories as acquired during the monitoring of the Li-Po battery set during the random as well as pre-programmed flights of a NEO hexacopter UAV by Ascending Technologies.A detailed description of the setup is given in (Mansouri, Karvelis, Georgoulas & Nikolakopoulos, 2017).The flying arena at Lulea University of Technology in Sweden where the tests took place has a volume of 5x5x3 m 3 .Two different type of flights were performed.Firstly, the UAV followed a predefined programmed trajectory, while in the second case, a pilot flew the UAV in the flying area; in both types of flights until the battery reached its EOD limit voltage of 14.2V.In the latter case, in an attempt to operate under nonconstant conditions, the pilot flied more aggressively and changed the altitude of the UAV more often.Figure 3 depicts the UAV trajectories in space in each case.The original sampling frequency used for the voltage data acquisition was 5 Hz.The minimum and maximum voltages for that specific power battery supply are 14V and 16.5V respectively.
The ten voltage discharge histories from actual indoor missions are shown in Figure 4.In all cases, the flight lasted until the battery voltage dropped to the EOD critical threshold of 14.2 V and an alert was triggered by the platform to immediately land the UAV.

Li-Po Diagnostics
As already mentioned, the case study consists of ten voltage discharge histories.The training dataset employs nine of them in order to estimate the NHHSMM's parameters θ* and keeps the tenth voltage discharge history as the testing diagnostic/prognostic dataset.
The Bayesian Information Criterion (BIC) was employed to estimate the optimum number of health discharge states which was found at N=4.Furthermore, the transition diagram allowed soft and hard transitions and the Weibull distribution was selected for the sojourn time.The voltage data quantization was achieved through clustering via a kmeans algorithm and under a monotonicity objective function, which can be utilized using the Mann-Kendal criterion (Eleftheroglou, Zarouchas, Loutas, Alderliesten & Benedictus, 2018).As a result, the number of clusters was found equal to V=25.
Figure 5 present the left outlier's SOH estimations, i.e.Battery 9, and Figure 6 the SOH estimation of Battery 1 which is a non-outlier case.The choice of presenting the results of Battery 1 was random, similar results were obtained for the other batteries.Battery 9 as already mentioned is the left outlier in the sense that is starting with a lower voltage and reaching the EOD critical limit quite earlier than the rest test cases.The discharge history of Battery 9 has not been included in the training process.Consequently, the minimum discharge time of this training data set is 9,857 min while Battery 9 discharge time is 5,736 min, see Figure 4.This way we are able to assess how the SOH/RUL methodology perform in such an extreme case.The SOH estimations can verify that this diagnostic measure can be successfully used for online discharge diagnostics since they have a non-decreasing monotonic trend over the Li-Po batteries lifetime.In case of the left outlier the SOH estimations are interesting since the sojourn time of the health/charge state one is very short, in comparison to sojourn time of Battery 1 SOH, Figures 5 and 6, and as a result this observation can be a first alarm regarding this battery.

Li-Po Prognostics
Respectively to Li-Po Diagnostics sub-section, Figures 7-8 present the mean RUL estimations for Batteries 1 and 9.The upper and lower bounds shown, describe the 95% confidence intervals providing an uncertainty quantification associated with the RUL estimation at each time point.It is evident, that the mean RUL predictions of a battery which is not outlier converge quite satisfactorily with the real RUL values (Figure 7).The confidence bounds manifest a desirable behavior, which is to converge as time progresses and the EOD threshold is reached.The performance of the NHHSMM in the left outlier dataset shown last in Figure 8 is interesting.Even though the battery in that mission discharged quite rapidly as compared to all other missions, and despite the fact that in this case's training dataset there is no similar behavior, the NHHSMM manages to capture the trend relatively well although it overestimates RUL at all times.It's interesting to note that despite the very bad initial predictions, as time passes predictions tend to improve.

CONCLUSIONS
In this paper, the challenging problem of estimating a Li-Po battery's state of health (SOH) and estimating the remaining useful flight time of a UAV are addressed.We propose the NHHSMM, which is a data-driven model, to tackle the aforementioned problems.We employ the voltage of the battery cells, as a feature, in order to train the NHHSMM and estimate the battery's SOH and RUL.Besides the mean RUL estimations, upper and lower bounds quantify the uncertainty associated with each point prediction.Detailed results are presented for two different batteries and the robustness regarding the diagnostics/prognostics is verified even in an outlier flight with a rapidly discharging battery.
The same methodology can be easily expanded in the diagnostics and prognostics of other power units as well e.g. in electric cars etc.

Figure 3 .Figure 4 .
Figure 3. Visualization of the trajectories followed during the UAV flights with predefined path (a) and manual flight (b)