Failure Mode Investigation to Enable LiDAR Health Monitoring for Automotive Application

Light Detection and Ranging (LiDAR) sensors are critical components of the perception system and play a significant role in enabling fully autonomous driving. Given that LiDARs have a higher failure rate than other sensors, such as camera and radar, it is crucial to monitor the health of this component to increasing the availability of autonomous driving features. Such a health monitoring system can additionally provide cost-effective maintenance for retail and fleet, improve the service experience of retail customers, and ensure the fidelity of the data produced by the LiDAR for engineering development. Since LiDAR is a relatively new technology, there is currently limited work in the area of LiDAR health monitoring. The failure modes and degradation behavior of these components have not been thoroughly studied in the literature for automotive applications. Therefore, this paper reviews LiDAR external and internal failure modes and their impacts on the perception performance. The external failure modes are categorized into multiple fault classes such as sensor blockage due to a layer of debris on the sensor, mechanical damage to the sensor cover, and mounting issues. The internal faults corresponding to LiDAR subcomponents such as transmitter, receiver or scanning mechanism, are explored for these LiDAR types: mechanical spinning, flash LiDAR, Micro-opto-electromechanical mirror LiDAR, and micromotion technology LiDAR. The failure modes of each subcomponent are also investigated to determine if they can be categorized as slow degradation or sudden failure. It is concluded that mechanical spinning LiDARs are expected to have higher failure rates than solid-state LiDARs. Both internal and external LiDAR failure modes can lead to reduced accuracy and reliability in detecting objects and obstacles, compromising the safety of autonomous driving systems, and increasing the possibility of collisions.


INTRODUCTION
Autonomous driving will allow everyone to have personalized land mobility (Brenner & Herrmann, 2018).One of the most important challenges in autonomous driving is to equip the vehicle with the level of perception at least similar to a well-trained human driver (Gruyer, et al., 2017).Various modalities of sensing have been studied and deployed, including camera, radar, ultrasound, and Light Detection and Ranging sensor (LiDAR) (Li & Ibanez-Guzman, 2020).LiDAR uses precise optical system to measure the amount of time it takes a laser pulse to return to the sensor.This time is converted to the distance from the object to the vehicle.Since laser devices and precise machining are needed to manufacture LiDAR sensors, the per-unit cost of LiDAR is usually a lot higher than other types of sensors used in autonomous driving (Rangwala, 2022).
When a mechanical scanning mechanism is involved in the LiDAR sensor, the moving parts are susceptible to degradation from wear and tear.Since the LiDAR sensor is usually a critical component in autonomous vehicles, its failure can lead to abortion of autonomous driving.To ensure the availability of autonomous driving, it is important to identify the imminence of LiDAR sensor failure early enough so the sensor can be serviced before it fails.Since LiDAR is a relatively new technology, there is currently limited work in LiDAR health monitoring.The failure modes and degradation behavior of these components have not been thoroughly studied in the literature for automotive applications.Therefore, it is valuable to understand the failure modes of the LiDAR sensors and develop methods to monitor their health.
In this paper, the failure modes of various types of LiDAR sensor will be reviewed, the categories of failures are introduced, and the impact of these failure modes to perception system will be discussed.

LIDAR FAILURE TYPES AND THE IMPACT ON THE PERCEPTION SYSTEM
In this section, we first classify LiDAR failure modes into external and internal failures.It is also determined if each failure mode is sudden or gradual and how it can potentially impact the perception performance.

LiDAR Failure Mode Classes
A failure mode is classified as internal if a failure occurs with a LiDAR's subsystem inside LiDAR, such as transmitter, receiver, processing unit, or sensor cover.Since a LiDAR sensor is delicate and requires specialized tools and skills to disassemble and assemble correctly, a service technician in a typical automobile dealership is unlikely going to be able to replace or repair any failed component inside the LiDAR sensor.In this case, the technician needs to replace the entire sensor unit.The failure modes that can only be rectified this way at a dealership are denoted as internal failures.
The following failure modes are categorized as an external: an external layer on the sensor window (e.g., dirt, water, ice, salt, snow) and a loose mounting bracket (Goelles, Schlager, & Muckenhuber, 2020).In this case, a technician can resolve the issues without disassembling the sensor.
For each internal and external failure mode, it is determined if the fault is sudden or gradual.For some failure modes, the failure happens in a short period of time, with almost no early sign of failure to be detected or monitored.These failures are denoted as sudden failure.Diagnostics must be implemented to capture this type of failures as soon as they happen.For other failure modes, the failure is a process.It can start as a degradation in performance as indicated by some metrics.The degradation can worsen over time until it matures as a failure.These failures are denoted as slow degradation.Since the early signs of this type of failure can be detected, prognostics strategy can be implemented to detect degradation.After degradation is detected, the vehicle owner can then be notified and preventive actions can be taken before the failure occurs.
The methods to detect these failures include investigating the point cloud or the meta data, both generated by the LiDAR.The point cloud gives a 3-dimensional view of the surroundings detected by the LiDAR.These points are defined by horizontal position, vertical position, and distance from the LiDAR.The meta data includes various signals from the internal components of the LiDAR.In some designs, the meta data also contains diagnostic flags to identify known failures.

Internal failure modes and their impact
The followings (shown in Figure .1)are typical LiDAR subsystems for which an associated fault is considered as a LiDAR internal failure mode: • Laser device: outputs pulses of light at the desired frequency, direction, and power level • Photodetector: detects the returning laser pulses • Microelectronics (processor): coordinates the subsystems and processes the returned pulses to meaningful information • Electrical connections: brings power to each subsystem and relays signals between the subsystems through connectors, splices, and clamp.
• Scanning mechanism: steers the light pulses to specific directions • Optical system: conditions and directs the light pulses by aligned lenses and mirrors • Thermal management: dissipates heat generated within the sensor into the desired direction at a desired rate • Enclosure: avoids unintended interaction between sensor components and its environment.The failure modes related to each of the above subsystems will be discussed in the following subsections.

Laser device failure
LiDAR sensors typically use laser devices as light sources.Lasers have very low beam divergence, making it convenient to steer its direction.Lasers also have a very narrow band of emission so that the light energy is concentrated into the wavelength range most beneficial to LiDAR sensing.
One sudden failure for a laser device is infant mortality.It can usually be captured during quality control in manufacturing testing.Slow degradation can also occur to laser devices.As a laser device is being used, its efficiency in converting electrical energy to optical energy does decrease.This slow degradation will gradually decrease the amount of light emitted by the laser device for the same amount of electrical energy pumped into the device.The reduction in light intensity will decrease the detection distance of the LiDAR sensor.The perception system may assign a lower confidence level for the data from LiDAR sensor.To maintain safe operation, the autonomous vehicle may need to slow down.
A laser device failure or degradation can be detected using the point cloud and the meta data.If the laser is degraded, the range will decrease.This will affect the identification of objects in the point cloud.Laser device failure can also be indicated by the voltage and current signals in the meta data.In addition, some diagnostic flags can indicate laser device failure.

Photodetector failure
Photodetectors used in LiDAR sensor need to have very high sensitivity to detect the backscattered light from the object.The gain of these detectors is usually very high.
As the photodetector is being used in a LiDAR sensor, its conversion efficiency from optical energy to electrical energy can decrease over time.Decrease in detected signal strength will cause a decrease in detection distance, resulting in wrongful identification of objects.As discussed in the previous section, a decrease in detection distance may require the vehicle to slow down to maintain safe operation.As a perception system will identify objects based on multiple signals including the returned signal strength, a drop in photodetector opto-electronic efficiency can decrease the accuracy of object identification.
In addition to the slow degradation in gain, the noise of the photodetector in a LiDAR sensor can also get worse over time.The increase in noise can increase the chance of false detection: a returned signal from a location is detected when no object is at that location.False detection can make the autonomous vehicle behave unreasonably as it steers away from an object which is not there.It will also complicate route planning unnecessarily as the vehicle wants to avoid objects that are not present.
It is important to highlight that, apart from the persistent faults discussed earlier, intermittent degradation in the reflected signal (due to scattering and saturation mostly) and subsequent performance deterioration can be attributed to adverse weather conditions, even in the absence of any faults directly associated with the photodetector (Wallace, Abderrahim, & and Gerald, 2020).The detected object properties (reflectivity, temperature, etc.) can also affect the strength of the reflected signal and consequently the LiDAR performance (Whiteman, 2003).This failure can be detected by looking at the point cloud, as there will be missing points and a lower range, which will affect object detection.The meta data will also provide insights about a photodetector failure through the voltage and current signals, and diagnostic flags provided by the LiDAR.

Microelectronics failure
There are multiple microelectronic chips working together inside a LiDAR sensor.Examples include oscillators synchronizing the chips, communication modules passing information through various subsystems, and processors transforming signals to useful information.
Usually, these chips come with numerous built-in diagnostics to detect failures or problems within the chip.
Over time, transistors in the chips age and degrade the timing accuracy of the chip (Taddiken, Hellwege, Heidmann, Peters-Drolshagen, & Paul, 2016).Since not all chips inside the sensor are made from the same combination of materials and structure, their timing accuracies will not degrade at the same rate.Therefore, the synchronization among the chips will become worse, causing detection frames to be dropped.This is classified as a slow degradation.As the number of dropped frames increases, the frame rate feeding into the perception system will also drop.Consequently, the trajectory of faster moving objects may not be detected.This degradation in perception will make object trajectory prediction unreliable and route planning irrelevant.Eventually, the autonomous vehicle will become unsafe to operate.
Microelectronics degradation can be detected using the meta data provided by the LiDAR.This degradation will cause the frame rate to decrease but will not affect the point cloud until there is a complete failure of the microelectronics.As the microelectronics degrades, it will eventually fail.Then, the LiDAR will not produce a point cloud at all.

Electrical connection failure
The connections of power supplies to the subsystems usually involve connectors, clamps, or splices.
If the grade of material or process used for these electrical connections are not robust enough for automotive application, these connections will degrade quickly.Over time, as the vehicle vibrates in various directions due to road conditions, these connections can become loose.Eventually, the continuity can become broken, causing the module to lose power.However, there are no indicators to represent a degraded connection, since power is still being supplied to the module.When this failure occurs, it results in a sudden loss of communication among the modules of the sensor.
When a connection fails, the sensor will usually fail to operate.The perception system will get no information from the LiDAR sensor.Since the LiDAR sensor provides unique distance information to the perception system and there is usually no redundancy from other sensors, perception may not be able to function correctly when a LiDAR sensor stops working.When this type of failure occurs, the LiDAR cannot produce any point cloud and perception cannot detect any object in the surroundings of the vehicle.When this happens, autonomous driving may need to be aborted.
An electrical connection failure cannot be detected using the point cloud, as it can cause no point cloud to be produced.This failure can be detected from the meta data information such as voltage, current, and diagnostic flags.

Scanning mechanism failure
To cover a wide field of view, the low-divergent laser beam is steered to various directions.For this purpose, a scanning mechanism is required which can be categorized into three primary groups: mechanical rotation, MEMS mirrors, and Micro Motion Technology (MMT) (Ghazinouri, et al, 2022).
Since multiple moving parts are involved in the scanning mechanism, the mechanism is usually more susceptible to failure due to wear and tear.This is a slow and continuous degradation process as the LiDAR sensor operates over time.The impact to perception can include reduction in frame rate, field of view, and resolution.
As the mechanically moving parts wear and tear, the friction among the moving parts can increase, making the mechanism harder to drive.The scanning rate also decreases which causes the frame rate to decrease.As discussed in the previous section, a decrease in frame rate will decrease the chance of detection of fast-moving objects.This change in frame rate will cause a loss of synchronization between the LiDAR and other sensors.This makes it difficult to fuse information from multiple sensors.The prediction and route planning ability will also be compromised, and autonomous driving will eventually need to be aborted.
As the friction in the scanning mechanism increases, the mechanism may not be able to steer the laser beam to the extreme corners of the intended field of view.If the friction change is not uniform, the scanning mechanism can be biased and miss some parts of the field of view.In these cases, even though other sensors may have correctly detected objects in that part of the field of view, the perception system will have less confidence in those objects detected.
As the scanning mechanism slows down, it can lose synchronization with the laser firing and detection subsystem.The field of view will be probed less densely than intended.The overall resolution from the LiDAR sensor will decrease.Consequently, the detection distance will decrease for the same object, or the size of the smallest object detectable will increase for the same detection distance.
This failure can be detected by looking at the point cloud of the LiDAR.If the scanning mechanism degrades, the overall resolution can be impacted, as well as the detection distance.Both issues can be identified from the point cloud.A scanning mechanism failure can also impact the field of view of the LiDAR, decreasing the horizontal and vertical ranges of the point cloud.

Optical system failure
Optical elements such as lenses and mirrors are usually used inside a LiDAR sensor to condition and steer the outgoing laser pulses and the returning backscattered light pulses.These optical elements need to be precisely aligned with the laser devices and photodetectors, so the outgoing light pulse can reach the corners of the field of view, and the returning light pulse can be focused onto the intended photodetector.
As the vehicle vibrates in various directions due to road surface conditions, the optical system can become misaligned.This is usually a slow degradation process.The misalignment usually becomes worse over time.Eventually, the outgoing light pulse may be biased to probe only a portion of the field of view, missing the remainder.Meanwhile, the returning light pulse may not be focused onto the intended detector, and the amount of light on the intended detector can fall significantly below the detection threshold, causing an object to be undetected.Perception will need to reconcile with other sensors to decide if there is any object at that location.If the missed detection rate increases, the autonomous driving will become unsafe.
An optical system failure can be detected by analyzing the point cloud.If the optical system becomes misaligned, the point cloud field of view will degrade.The resolution of the point cloud can also be affected if the optical misalignment causes additional scattering.

Thermal management failure
All components consuming electricity in the LiDAR sensor produce heat.The performance and efficiency of the microelectronics, laser device, and photodetector all depend on their ambient temperature.In particular, the intensity and quality of light emitting from a laser device decreases sharply as ambient temperature increases.The noise of the photodetector increases significantly as ambient temperature increases.To maintain the temperature inside the LiDAR sensor enclosure at an acceptable range for these components, most of the heat-generating components are thermally connected to heat sinks.The heat from the heat sink will then be directed outside of the sensor enclosure.
Over time, the thermal connection between these heat generating components and their heat sinks will become loose.This can be caused by mechanical misalignment due to vehicle vibration, or degradation of the thermal paste due to extreme temperature swing from winter to summer.These are both slow degradation processes.When the heat transfer efficiency drops sufficiently, the temperature inside the sensor will increase rapidly to an inoperable temperature.This results in a decrease in efficiency of the heat generating components as they operate at high temperatures, causing the laser to emit less light than expected.As discussed in the previous section, the consequence is a decrease in detection distance.The photodetector will also be noisier and produce more false detection.If the temperature continues to rise, the components must be powered down to prevent permanent damage.If this high-temperature shut down of the LiDAR sensor happens often, the availability of the perception system and, hence, the autonomous driving system will decrease.
A thermal failure can be detected by the temperature signal and diagnostic flags in the meta data.

Enclosure failure
Usually, all electricity-consuming components of a LiDAR sensor are placed inside an enclosure to prevent unintended interaction with the environment.For ingress prevention, there are usually gaskets around the enclosure.While the enclosure itself may last for a long time, the gaskets can degrade over time when experiencing significant temperature fluctuations throughout the year.This is a slow degradation process.When ingress protection is compromised, dust and moisture will enter the enclosure.The ingress is especially significant when the LiDAR sensor is mounted outside the vehicle.A mechanical damage such as deformation, crack, hole, or scratch can also suddenly occur.
As moisture accumulates on the components inside the enclosure, it accelerates corrosion of exposed metal components.If corrosion affects the scanning mechanism, it causes the field of view, resolution, and frame rate of the perception system to be compromised.
As dust accumulates on optical components inside the enclosure, it will increase scattering of the light to unintended location.For an outgoing laser pulse, increased scattering inside the sensor will decrease the intensity of light exiting the sensor, which will decrease detection distance of the perception system.For the returning light, increased scattering will decrease the amount of light reaching the photodetector, which will decrease the detection rate of the sensor.
This type of failure can be detected by analyzing the point cloud.An enclosure issue will affect the field of view, resolution, and frame rate.

External failure modes and their impact
The following are the subsystems for which the associated fault is considered as external: • Mounting device: secures the sensor to the vehicle at a specific location and orientation • Sensor window: allows light pulses to go out of and return to the sensor.
The failure modes related to each of the above subsystems will be discussed in the following subsections.

Mounting device failure
The LiDAR sensor can be mounted inside or outside the vehicle.In either case, a mounting device is used to fix the position and orientation of the LiDAR, relative to the vehicle frame and other sensors being used for perception.The correct alignment between the LiDAR and other sensors is crucial to enable the fusion of information from multiple sensors for the same scene.Successful fusion will increase the confidence of object detection and identification by the perception system.This will, in turn, increase the reliability of prediction and effectiveness of route planning.
Over time, the mounting device may become misaligned due to vehicle vibration from road surface conditions.This is usually a slow degradation process.As misalignment worsens, the perception system will have increased trouble with detection, as the location of the object detected by the LiDAR sensor diverges from that detected by other sensors.Hence, the reliability of object identification by the perception system will decrease.
Nevertheless, this is an external failure.When a vehicle with LiDAR sensor mounting problem arrives at a service department of a dealership, the technician can perform alignment of the LiDAR sensor without disassembling the LiDAR sensor.This failure is relatively easy and low-cost to recover.
A mounting device failure is difficult to detect using only the point cloud and the meta data, as this failure causes misalignment between the LiDAR and other sensors.It may be possible to exploit fusion methods between the LiDAR and other sensors in the perception system.There is a multitude of fusion methods being developed for various applications (Zhong, et al., 2021).These methods can potentially help to determine if the sensors in the perception system are aligned.

Sensor window failure
A LiDAR sensor has a window to allow outgoing light to exit the sensor and returning light to enter the sensor.It typically filters out any light with wavelengths outside the range used by the LiDAR.For a LiDAR mounted outside the vehicle, this window will be exposed to the outside environment.However, for a LiDAR mounted behind the windshield, only the windshield will be exposed to outside environment.
Whether the vehicle is being driven regularly or parked in an indoor garage most of the time, dust or dirt will accumulate on the sensor window or windshield over time.In winter conditions, snow and salt can also accumulate on this surface.These accumulations will increase scattering for both outgoing and returning light pulses.As discussed in the previous sections, increased scattering will decrease detection distance and increase the chance of undetected objects.
As the accumulation grows thicker, it will eventually block all outgoing and returning light pulses.The corresponding portion of the field of view of the sensor will be blocked.Within that blocked field of view, the perception system will have less confidence on the objects detected by other sensors.It will also have no distance information about those objects detected.
Nevertheless, this is an external failure.When a vehicle with a LiDAR sensor window problem arrives at a dealership for service, the technician can clean up the accumulation without disassembling the LiDAR sensor.Furthermore, the driver can check and clean the windshield or sensor window regularly to ensure no significant accumulation is present.Although this failure is a slow degradation, it can be rectified easily.
A sensor window failure can be detected by analyzing the field of view in the point cloud.When the sensor window is blocked, the corresponding portion of the field of view is blocked as well.
The failure modes, types, and impact to perception system are summarized in Table 1.

FAILURE MODES IN DIFFERENT TYPES OF LIDAR
Although most of the failure modes discussed in the previous sections are common to most LiDAR designs, some designs are immune to certain failure modes while other designs are more susceptible to other failure modes.
LiDAR sensors with rotating mirrors are a popular design among commercial LIDAR sensors (Royo & Ballesta-Garcia, 2019).Although they are relatively more mature among LiDAR technologies, they are more susceptible to failures due to the increase of moving parts.They are typically mounted on the top of the vehicle, outside of the cabin.This arrangement exposes the sensors directly to the elements, where the rotating mechanism can be interfered by ice or dirt accumulation.The sensor window is also more likely to be damaged or blocked by accumulation of sand, salt, snow, or dirt.However, this type of LiDAR provides a 360 o field of view in the azimuth direction, offering the perception system an instantaneous overview of the vehicle surroundings.
Contrary to a rotating-mirror LiDAR, the flash LiDAR has no scanning mechanism.Therefore, it is immune to the failure caused by moving parts.However, it may not have enough detection range or field of view due to laser eye safety compliance and practicality of light source power (McManamon, Banks, Beck, Huntington, & Watson, 2016).
Microelectromechanical systems (MEMS) have also been used in LiDAR sensors (Holmström, Baran, & Urey, 2014).MEMS has arrays of very small mirrors, with sizes in the millimeter range.Each of these mirrors can be tilted at various angles.By coordinating the tilting of these mirrors, a field of view can be scanned.Since each of these mirrors can be tilted independently, the field of view can be programmable.Since this type of LiDAR has no spinning mechanism, it has significantly less wear and tear due to macroscopic spinning action (Ghazinouri & He, 2023).However, the small area of each mirror may not be able to handle high intensity laser pulses, which limits the detection distance of the sensor.

CONCLUSION AND FUTURE WORK
In this paper, failure modes for typical LiDAR sensor subsystems are discussed, with reasons of failure and their impacts to the perception system.

Figure 2
Figure 2 presents a qualitative comparison of failure modes based on their impact and occurrence rate.It's important to note that the occurrence rate and impact can vary widely across different types of LiDAR.

Figure 2 -
Figure 2-Qualitative comparison of occurrence rate and impact of lidar failure modes.

Table 1 .
It is concluded that mechanical spinning LiDARs are susceptible to more failure modes than flash LiDARs.Both internal and external LiDAR failure modes can lead to reduced accuracy and reliability in detecting objects and obstacles, which compromises the safety of autonomous driving systems and increases the possibility of collisions.Health monitoring strategies can only be designed for the failures resulting from slow degradation.Summary of typical failure modes and their impacts for LiDAR sensor Global Technical Specialist for Vehicle Health Management.He also has served as a postdoctoral medical robotic researcher at Queen's university, senior automation engineer for industrial Siemens SCADA/DCS solutions, and senior mechatronics specialist at AUS mechatronics center.His research interests include autonomous systems and medical robotics.He has published numerous patents and articles in these areas, featured at IEEE transactions journals, and received several awards.