Probabilistic Safety Assessment in Composite Materials using BNN by ABC-SS

.


INTRODUCTION
Artificial Neural Networks have recently experienced an outstanding development, mostly due to their successful application to a wide range of fields, such as computer vision (Voulodimos, Doulamis, Doulamis, Protopapadakis, & Andina, 2018) or speech recognition (Arora & Singh, 2012). It is indisputable that they are changing our daily lives and will continue to do so, however, those algorithms are not always correct in their predictions and can make mistakes. This is natural and, in many cases, cannot be avoided given the inherent randomness of many process on earth (Hüllermeier & Waegeman, 2021). It could then be stated that all predictions made by artificial neural networks are, in varying degrees, uncertain. Hence, quantifying such uncertainty can become critical depending on the importance of the subsequent decision making process (Ghahramani, 2015). Moreover, measuring and managing the degree of belief in the predictions play a major role in the prognosis field (Zhang, Liu, Zhang, & Miao, 2020;Niu, Wang, Liu, & Zhang, 2021;Lyu et al., 2021). Precisely, the current methods for identifying fatigue and its propagation in composite materials need to deal with a significant amount of uncertainty, mainly due to the complexity of the fracture processes present in these materials (Srinivasa et al., 2010).
While modern neural networks could provide relatively good predictions in this field, they are unhelpful if not paired with some notion of how certain those predictions are. Moreover, that is one of the reasons why these materials are not used in the aerospace industry on a large scale, as it is difficult to assess the degree of belief in the predictions about the remaining useful life of the material. The so-called Bayesian Neural Networks, such as Hamiltonian Monte Carlo (Benker, Furtner, Semm, & Zaeh, 2020;Levy, Sohl-dickstein, & Hoffman, 2018), Variational Inference (Graves, 2011;Hoffman, Blei, Wang, & Paisley, 2013;Wang, Bai, & Tan, 2020) (Bayes by Backprop (Blundell, Cornebise, Kavukcuoglu, & Wierstra, 2015;Jia, Yue, Yang, Pei, & Wang, 2020)) or Probabilistic backpropagation (Hernandez-Lobato & Adams, 2015), have provided good results when quantifying the uncertainty in different applications. However, they have parametric weights, predefined cost/likelihood functions and their learning process is based on the backpropagation algorithm (Rumelhart, Hinton, & Williams, 1986). All that translates into a rigid quantification of the uncertainty, and a certain predisposition to problems such as instability or exploding/vanishing gradient. Contrariwise, BNN by ABC-SS (Fernández, Chiachío, Chiachío, Muñoz, & Herrera, 2022) has proven great flexibility to capture the uncertainty inherent in the observed data, thanks to its gradient-free nature, the non-parametric formulation of the weights and the absence of likelihood/cost function.
In this paper, BNN by ABC-SS is applied to an experiment of micro-crack propagation in carbon fiber composite materials, and compared against the state-of-the-art BNN. The predictions from those algorithms are then used in a probabilistic safety assessment. The probability of failure is calculated based on the quantification of the uncertainty obtained by each algorithm, with respect to a predefined failure threshold. The results obtained show the capacity of BNN by ABC-SS to accurately quantify the uncertainty in its predictions without restrictions and based on real observations, providing very valuable information about the potential failure of the material. This probabilistic prediction can become critical when evaluating the safety of an element (Pulkkinen & Huovinen, 1996), and of great importance when used for making decisions regarding maintenance. BNN by ABC-SS provides a new tool to navigate through the uncertainty inherent in safety assessments and management.

BNN BY ABC-SS
Artificial neural networks are used to perform a wide variety of tasks, such as making predictions about some target variables. However, those predictions are not always correct, and they can often be significantly imprecise depending on many factors, normally related to the quality of the training data. Therefore, there exists uncertainty about the accuracy of the predictions, just like nature is uncertain itself. It could then be agreed that, in those cases where the outputs of the ANN are used for a subsequent decision making process, quantifying the uncertainty or degree of belief is important (Gawlikowski et al., 2021). Bayesian Neural Networks are good at doing exactly that, given that they provide us with probabilistic predictions, comprising the most plausible values. Several types of BNN can be found in the literature, but Variational Inference (Bayes by Backprop), Probabilistic Backpropagation and Hamiltonian Monte Carlo have attracted the attention of the scientific community. However, they all include gradient descent to update the parameters of the neural network, and use a parametric formulation (often Gaussian) to define the the weights and/or the likelihood function, which leads to a rigid representation of the uncertainty (Ghahramani, 2015).
When ABC-SS (Chiachio, Beck, Chiachio, & Rus, 2014) is used as the learning engine, those drawbacks disappear, given its non-parametric weights, and the absence of likelihood function and gradient evaluation. Mathematically speaking, BNN by ABC-SS aims to find the posterior distribution of the weights w and bias b, based on a training data set D(x, y), and using the Bayes theorem as follows: where p (θ|D, M) is the posterior PDF of the parameters θ = {w, b} ∈ Θ ⊆ R d in model class M (architecture of the neural network), p(θ|M) is our prior knowledge or information, p (D|θ, M) is known as the likelihood function and p (D|M) is called the evidence.
, where the conditioning to the model class M has been omitted for clarity. This last equation shows that the posterior distribution of the parameters depends on the likelihood function, which can be unknown or simply intractable (Marin, Pudlo, Robert, & Ryder, 2012). The ABC method allows us to avoid the formulation of such likelihood function by selecting, as posterior samples, the pairs (θ,ŷ) ∈ S ⊆ Θ × O which satisfy thatŷ ∼ p (ŷ|θ) fall within a limited region around the data y given by B ϵ (y) = {ŷ ∈ O : ρ(η(ŷ), η(y))ϵ}, where the metric function ρ(·) evaluates the closeness betweenŷ and y using a vector of summary statistics η(·) (Fearnhead & Prangle, 2012). The posterior PDF of the parameters can now be defined as is the approximated likelihood function which takes the unity when ρ(η(ŷ), η(y)) ≤ ϵ, and 0 otherwise. In order to make this sampling process more efficient, the Subset Simulation method (Au & Beck, 2001) is used, which transforms a rare event simulation problem into a sequence of simulations with larger probabilities. Indeed, a sequence of nested regions S j , j = 1, . . . , ℓ are defined, such that S 1 ⊃ S 2 . . . ⊃ S ℓ = S, where S j = {(θ,ŷ) : ρ(η(ŷ, η(y))ϵ j }, and ϵ j+1 < ϵ j ∀j = 1, . . . , j. The interested reader is referred to (Chiachio et al., 2014) for further information about ABC-SS, and to (Fernández et al., 2022) for details about the implementation of BNN by ABC-SS.
The different stages of the proposed training method are illustrated in Figure 1. As per any other Bayesian process, the first step consists of obtaining N samples of the model parameters θ = {w, b} from the prior PDF p(θ|M), which in our case is assumed to follow a normal distribution N (0, 1). Then, each of those N samples of model parameters θ n are introduced in the ANN to carry out the forward pass and produce the outputsŷ n , using the training data set. The error incurred by those outputsŷ n are evaluated using the chosen metric ρ(η(ŷ, η(y)). In the next stage, the model parameters θ n are ranked depending on the value of this metric, the best P 0 N samples are selected to form the new posterior PDF p (θ|D, M), and the tolerance value ϵ is fixed based on the greatest error provided by the P 0 N samples. Finally, if the desired tolerance value ϵ has been achieved, the training process ends and the posterior PDF of the model parameters is fixed. Otherwise, new model parameters need to be resampled from the new posterior PDF, and the same process is followed iteratively until the desired ϵ is achieved or the maximum number of simulations ℓ is reached. The desired ϵ and/or ℓ need to be defined by the user in advance, depending on the level of precision required. Also, the number of samples N and the value of P 0 are selected based on the complexity of the ANN architecture, the more complex this is the more samples and lower P 0 are required.

EXPERIMENTAL FRAMEWORK
A probabilistic safety assessment of composite structures subjected to fatigue has been carried out. In this section, the experiment is described including how the data sets are prepared, what algorithms are used, the methodology to assess the probability of failure, and finally, the results are presented and discussed.

Fatigue in composite structures
Structural elements made of carbon fiber reinforced polymer (CFRP) present very good properties, even better than most metals. They are high performance heterogeneous materials with very high strength-to-weight ratios. However, it is still difficult to predict how they will behave under fatigue, as this process is partially unknown and subject to much uncertainty (Chiachío et al., 2015). Damage in composites typically comprises different modes (Talreja, 2008), such as intralaminar and interlaminar cracks, fiber-matrix debonding, fiber kinking and fiber pull-out among others. They can appear in isolation or in combination, resulting in a significant change in the structural performance of the element. This is the main rea-son why current physics-based models are not suitable, given they may work for specific forms of damage but not once an additional damage type appear. That uncertainty is responsible for the very limited applications of carbon fibre composite materials to aerospace engineering, where there exist high safety and reliability standards. Therefore, it seems sensible to use data-driven solutions that avoid the formulation of the different modes of failure, which are also able to quantify the uncertainty inherent in the fatigue process.
In this manuscript, four different BNN are used to predict the microcrack density in a CFRP laminate. The data consist of sequences of intralaminar micro-cracks density measurements for three different laminates with the same crossply ([0 2 /90 4 ] s ) layup. The data used are taken from the NASA Ames Prognostics Data Repository (CFRP Composites Dataset) and correspond to the laminates TD19, TD21 and TD22. For more details about the experiments and the data collected please refer to (Saxena, Goebel, Larrosa, & Chank, n.d.). These data come from a network of 12 piezoelectric (PZT) sensors using Lamb wave signals (Larrosa Wilson & Chang, 2012). For this study the dataset is designated as D(x, y), which comprises loading cycles as inputs x and micro-cracks density as observed outputs y. Measurements from the first pair of sensors in TD19 have been excluded from the training data set and used as test data. Also, both data sets have been normalized to take values in the range [0, 1]. For the comparison exercise, the different BNN are asked to predict the micro-crack density (ŷ) given the loading cycles x as inputs.
Once the predictions from the different BNN about the microcrack density have been obtained, and the uncertainty has been evaluated by each of those algorithms, a probabilistic safety assessment is carried out. That way, we can assess not only what we know, but also measure what we do not know.

Baseline Algorithms and metrics
As explained in Section 3.1, four different algorithms are used for this experiment. The neural network structure is common to all of them, comprising two hidden layers with 5 neurons each, and one output layer with one neuron (microcracks density). The hyperparameters have been chosen individually for each algorithm as follows: • BNN by ABC-SS: A BNN trained with Algorithm 1 of (Fernández et al., 2022), adapted with a while loop and σ j = σ 0 p. The hyper-parameters used are P 0 =0.1, N =100,000, σ 0 =0.75, p=0.58 and tolerance value ϵ=0.025. The value of σ and p are chosen by a trial and error process, very much like the learning rate in standard backpropagation. The activation function for the hidden units is ReLU.
• Variational Inference, Bayes by Backprop (BBP) (Blundell et al., 2015): A BNN with the baseline ar-chitecture, trained with an open source algorithm 1 implemented in Keras (Chollet et al., 2015). The hyperparameters have been chosen based on those found in the original code with lr = 0.001, epochs = 100, 000 and 500 samples. The activation function for the hidden units is LeakyReLU. The performance of the algorithms is evaluated using the first sensor in TD19 as test data. Their capacity to quantify the uncertainty is graphically assessed by the Inter Quantile Range (IQR). Finally, a safety assessment is undertaken in probabilistic terms, which is then cross-validated with the observed data to evaluate its consistency.

Probabilistic Safety Assessment
Safety is critical in aerospace engineering, and it is the primary driver for all decisions about materials, designs and technologies to be implemented. As discussed in Section 4, the behaviour of composite structures under fatigue, and the interaction between their different modes of failure, are not yet well understood, which limits their implementation. Therefore, a reliable evaluation of their probability of failure is an important step towards a large scale application.
The proposed methodology starts by setting a failure threshold for the target variable, micro-crack density in our case study. This a value which, if exceeded, the composite structure will perform below a required safety standard, and does not necessarily mean material breakage. In this context, it is case specific and may differ depending on the particular application. In the experiment described in this manuscript, the threshold has been set to 0.8 (normalized). Next, the different BNN are trained, so we can make predictions on the test data. These neural networks are probabilistic by nature, so their outputs are not deterministic values but a density function. The number of samples that we draw from this output is chosen by the user, and in our case they can be found in Section 3.2. Finally, the probability of failure, being 0 very unlikely and 1 very certain, is calculated based on the proportion of samples that fall beyond the failure threshold, as follows: The experimental data is also used to calculate the observed probability of failure, so it can be compared against the predictions obtained from the Bayesian neural networks and check if they are consistent.

Results and Discussion
The performance of the Bayesian algorithms described in Section 3.2, evaluated on the CFRP Composites Dataset from NASA Ames Prognostics Data Repository, was discussed in Table 1 of (Fernández et al., 2022), where the accuracy and stability of BNN by ABC-SS was demonstrated. The capacity of the algorithms to capture the uncertainty inherent in the training data is graphically assessed in Figure 2. It can be seen that, while the mean predictions of PBP and HMC might be accurate, they fail to accurate capture the variability of the training data, resulting in an unrealistic quantification of the uncertainty. Contrariwise, BNN by ABC-SS seems to adapt significantly well to the training data, enclosing the vast majority of the data points. And that flexibility to capture the plausibility of the outputs, mostly thanks to the non-parametric formulation of the weights and the absence of likelihood function, is what makes BNN by ABC-SS suitable for use in probabilistic safety assessments.  The probability of failure has been calculated for the last cycles of the experiment, following the methodology explained in Section 3.3, and the results are shown in Table 1. It can be seen that BNN by ABC-SS provides the closest probabilities to the observed data. This is clear when comparing the average difference (root mean squared error) between the probabilities given by the different algorithms and the observed data, which are: BNN by ABC-SS (0.15), HMC (0.29), PBP (0.31) and VI (0.24). The results in Table 1 have also been illustrated in Figure 3, where we can see that the green line is the best fit to the observed data. Moreover, those data suffer from noise, which is most likely responsible for the negative slope in some parts of the dashed grey curve. This issue is solved by all four algorithms, as they are monotonically increasing, however, HMC and PBP seem to provide a more simple approximation, going from 0 to 1 in just a few loading cycles. Figure 3. Evaluation of the probability of failure (0 to 1), based on the predictions made by the different Bayesian Neural Networks. The threshold for plausible failure was set at 0.8 micro-crack density (normalized). .
Finally, the predictions made by BNN by ABC-SS during the last cycles of the experiment are shown in Figure 4 (green PDF), and compared against the given data (grey PDF). While the shape of those density functions are not a perfect match, the overall estimation about the probability of failure, mean-ing the area of the PDFs located to the right of the threshold line (red), are acceptably accurate. Again, this is thanks to the flexibility of BNN by ABC-SS to capture the uncertainty and variability found in the data. Figure 4. Probability density function (PDF) of predictions made by BNN by ABC-SS at different loading cycles. Those predictions, shown in green, are compared against the observed data, which are shown in light grey. The red line represents the failure threshold, and the probability of failure is given by the area of the PDF located to the right of this line. .

CONCLUSIONS
Composite structures, such as carbon fiber reinforced polymers, present very good properties and potential applications in the aerospace filed. However, there exist a lack of knowledge regarding their behaviour and performance when they are subjected to fatigue, and therefore, it is difficult to predict their remaining useful life. Those gaps in the current scientific knowledge can be express as uncertainty, which can be measured. Whilst there are many different methods to deal with the uncertainty, BNNs have demonstrated a good performance and are increasing in popularity within the scientific community.
Four different Bayesian Neural Networks have been applied to the CFRP Composites Dataset from NASA Ames Prognostics Data Repository, so their capacity to capture the uncertainty could be evaluated. Then, a probabilistic safety assessment was carried out based on the predictions made by the algorithms. BNN by ABC-SS provided the best results, demonstrating flexibility to capture the variability in the data. Thereby, its predictions about the probability of failure approximated the observed data significantly well.
While there doesn't exist a unique physics-based model to explain the mechanisms of failure in composite structures, Bayesian Neural Networks, and specially BNN by ABC-SS, could become a useful tool to quantify the uncertainty inherent in the behaviour of composite materials. Moreover, their predictions can be used in subsequent probabilistic safety assessments, which in turn helps to make better informed decisions regarding maintenance, or the potential replacement of the structural element.