Physics-Informed Deep Learning-Based Approach for Probabilistic Modeling of Degradation

Deep learning (DL) models have gained significant popularity for the prognostics of systems experiencing degradation. However, there are two major concerns with such models. Firstly, they require a substantial amount of training data due to their large number of parameters. Secondly, they disregard the underlying physics and solely fit the available data, leading to potentially weak generalization capabilities when faced with unseen out-of-distribution data in the field. This study aims to tackle these challenges by incorporating the underlying physics of degradation into DL models. The objective is to develop a novel DL-based approach in conjunction with Bayesian filtering, enabling physics-informed probabilistic life prediction for systems subject to environmentally induced degradation. The proposed framework consists of two main components: physics discovery and degradation prediction. The former involves identifying the dominant stress agents and formulating the underlying physics of degradation. The latter predicts the degradation of the system by incorporating the discovered physics into a DL model. It is expected the results indicate that by combining data-driven DL with physics-based insights, more robust and reliable life predictions can be achieved, addressing the limitations of DL approaches. This framework holds promise for enhancing decision-making processes related to maintenance strategies in various industries.


PROBLEM STATEMENT
Physics-based models are powerful tools that can be used to predict the degradation of systems based on their fundamental underlying principles.These models are typically developed empirically in a controlled laboratory environment and have been widely used for prognostics in various engineering systems.One of the most well-known examples of a physics-based model is the Paris law (Paris, Gomez, Anderson, & Pelloux, 1961), which establishes a relationship between the rate of crack propagation and stress intensity.However, the applicability of these models in realworld conditions can be limited due to their inability to capture complex interactions when there are multiple environmental factors.
Therefore, there is a need for more robust models that can account for the complexity of real-world conditions and accurately predict the lifetime of systems in a more general setting.In recent years, deep learning (DL) models have emerged as an increasingly prominent approach to address this challenge.Unlike physics-based models, DL models possess the ability to consider all relevant environmental factors and their complex interactions, owing to their capacity to effectively fit high-dimensional feature spaces.This capability makes DL models more applicable for predicting systems degradation in real-world scenarios.
However, employing DL models encounters two significant challenges: data scarcity and interpretability.These challenges cause limitations on the effectiveness and practicality of DL models for the prognostics of systems.DL models are characterized by a large number of trainable parameters, and as a result, they typically require large datasets to effectively train a prognostic model.However, collecting sufficient data from degradation is often timeconsuming due to the slow nature of degradation mechanisms such as corrosion, fatigue, and creep.As a result, data scarcity is a common challenge when training DL models for prognostics.
Furthermore, DL models are purely data-driven approaches that just fit the data without any understanding of physics.The purely data-driven nature of DL models as well as the lack of interpretability in such models may cause them to unknowingly violate the underlying physics of degradation.This violation can consequently lead to poor generalization when these models are deployed in real-world settings, where they may encounter out-of-distribution field data that was not present during training.
Both the issues mentioned above, namely data scarcity and weak generalization, can be effectively addressed by integrating the physics of degradation into DL models.Incorporated physics imposes constraints on the search space of model parameters, leading to more efficient training with limited data.In addition, this integration forces the model to adhere to the physics besides fitting the data, thereby enhancing its ability to generalize well beyond the training data.
However, the extent to which physics can aid DL models varies, as physics can be a simple understanding of the degradation phenomena (e.g., degradation irreversibility) or more sophisticated relationships discovered from data.That represents the major issue in using physics-informed models for the prognostics of systems since for the complex process of degradation in real-world conditions, where multiple stresses impact degradation synergically, the underlying physics is usually highly unknown.

EXPECTED CONTRIBUTION
This study proposes an integrated approach to discover the underlying physics of degradation in a complex system and incorporate it into a predictive model to improve the robustness and accuracy of degradation predictions.By integrating the discovered physics into the predictive model, the training efficiency of the model can be improved significantly even when data availability is limited.
In addition, the study proposes the utilization of Bayesian filtering to leverage prior knowledge which may exist from a potentially available empirical physics-based model for the considered system.By employing Bayesian filtering, the approach not only benefits from the potential prior knowledge but also can quantify the associated uncertainty with the predictions.
Figure 1 shows the different components of the proposed approach for the prognostics of systems under degradation.The approach is structured into three levels.At the first level, the underlying physics of degradation is discovered using previously collected data from either a similar system or short-term tests within the range of operating conditions.The dominant environmental factors  impact the degradation rate   non-linearly (Equation 1) and this non-linear function  can be determined using the physics discovery model.The dominant environmental factors can be identified using feature importance measurement methods in machine learning such as random forest and permutation.The three potential methods that can be employed for discovering the underlying physics of degradation (i.e., function ) are based on Deep learning (Raissi, Perdikaris, & Karniadakis, 2019), symbolic regression (Sun, Ouyang, Zhang, & Zhang, 2019), and sparse regression (Brunton, Proctor, & Kutz, 2016).
At the second level, the predictive model not only fits the training data but also follows the physics, which was discovered at the first level.This integration enables the model to make predictions that are consistent with the underlying physics.In Equation 2, cost function  for the predictive model is presented, comprising two terms: 1)   for fitting the data and 2)  ℎ for following the underlying physics learned by the physics discovery model.The model is trained using the labeled data and   represents the error between the estimates (i.e., the model's output) and the observed (true) values. ℎ acts as a constraint in the optimization process of the model's cost function, with a scalar parameter λ serving as the weight for this constraint.
Figure 1.Proposed approach At the third level, when a previously developed empirical model exists for a considered target system, it can be leveraged as prior knowledge.Through Bayesian filtering, the estimation of the empirical model can be updated by incorporating the output of the predictive model as observations.This integration allows for the fusion of the physics-based and data-driven models, harnessing the strengths of both approaches for more accurate and robust predictions.In addition, this approach enables the quantification of uncertainty in the predictions, which is necessary for facilitating robust decision-making processes.
To accomplish this, the Particle Filtering technique is proposed, which is a recursive Bayesian approach based on a sequential Monte Carlo method.

RESEARCH PLAN
The initial step of this study is to conduct a comprehensive literature review to understand the current state of the art.The proposed approach can be further divided into three main parts, all of which can be initiated concurrently: 1. Development of a physics discovery model: This part involves the exploration of various methods for discovering the underlying physics from data.A comparative analysis is necessary to evaluate the performance of these methods within the proposed framework.Although the approach's performance can be evaluated and optimized using simulated datasets, to validate its effectiveness it is essential to conduct experiments using benchmark datasets.Therefore, it is crucial to conduct a comprehensive search to identify suitable benchmark datasets for validating the proposed approach.

Work Performed
Some progress has been made in the first two parts of the proposed approach, and based on simulated datasets, it has been shown how the incorporation of underlying physics into the predictive model can enhance the accuracy of degradation predictions.Figure 2 illustrates a comparison of the actual degradation intensity with the estimated degradation intensity for both a purely data-driven predictive DL model and a physics-informed predictive DL model.The results clearly demonstrate that integrating the underlying physics into the predictive model improves the accuracy of degradation predictions significantly.

Remaining Work
While the physics discovery and predictive models have been partially developed for a noise-free simulated dataset, it remains unclear whether these models perform well under real-world conditions with noisy data.Further investigations are required to address this concern.
In the case of the predictive model, it is important to quantify the extent to which incorporating physics improves its performance when dealing with out-of-distribution data.Also, a sensitivity analysis of model parameters should be conducted to assess their impact on the model's performance.
The remaining work also involves the development of Bayesian filtering, which enables the fusion of the predictive model with an empirical physics-based model.
Finally, as mentioned earlier, validating the proposed approach using an appropriate dataset is essential to demonstrate its effectiveness.This task should be prioritized as it provides a means to evaluate and verify the performance and reliability of the approach in practical scenarios.

CONCLUSIONS
The objective of this study is to propose a novel approach that addresses the issues of data scarcity and weak generalization for out-of-distribution data in DL models by integrating the physics of degradation.To enhance the performance of the predictive model, advanced methods for discovering sophisticated physics will be employed, leveraging state-ofthe-art techniques.The outcome of this research work will contribute to the development of efficient predictive maintenance strategies in various industries.

2.
Development of a predictive model: In this part, a predictive DL model is trained to predict degradation intensity by fitting the training data while adhering to the discovered underlying physics.3. Development of a Bayesian filtering technique: This step focuses on implementing a Bayesian filtering approach to integrate a physics-based model with the DL model and quantification of associated uncertainties.

Figure 2 .
Figure 2. Actual vs. estimated degradation intensity for a) purely data-driven predictive model and b) physicsinformed predictive model