Requirements and data integrity considerations for diagnostics testbeds

The process of generating high quality data for the test and evaluation of diagnostic and prognostic algorithms is still of high importance to the Prognostics and Health Management (PHM) research community. To support these efforts a testbed has been designed, manufactured and commissioned. It has specifically been designed in order to replicate several component degradation faults with high accuracy and high repeatability. This paper documents the design, requirements and the data integrity elements of this benchmark hydraulic system. This document consolidates the process of designing diagnostics testbeds as at present there is a lack of literature on how diagnostics testbeds should be built and is intended to serve as a starting point and quick reference guide for engineers and researchers intending to design and develop a testbed to test and validate PHM applications. The first part of this paper highlights design requirements for all the design aspects for such testbeds with great consideration for industry standards and best practices covering the achievement of electromagnetic compatibility (EMC) and noise mitigation, as well as operators’ safety and equipment protection. The second part of the paper put great emphasis on data integrity elements of the data generated by this testbed (describing the system under healthy and faulty conditions) before it is actually used for system characterization or by diagnostics and prognostics algorithms.


INTRODUCTION
Optimized and reliable PHM solutions contribute to increased availability and reduced maintenance costs for high-tech high-value systems. As a result, the software market provides a significant number of diagnostic and prognostic software packages for design and implementation of PHM solutions. Assessment and evaluation of such software can be a lengthy and painstaking project on real machines due to the large number of uncontrollable parameters that affect the acquired data. An alternative approach, adopted here, is to develop a demonstrator platform in which known faults can be introduced, accurately and repeatable, and their consequences monitored. In this work, a demonstrator platform capable of producing high quality data representing normal and abnormal scenarios has been built. The data can be incorporated into a consistent framework for evaluating diagnostics and prognostics algorithms.
The designed testbed represents a fuel system and its associated electrical power supply, control system and sensing capabilities. This testbed should be capable of simulating a wide range of fault types/profiles on components and instrumentation to produce benchmark datasets to evaluate and assess diagnostic and prognostic tools. Single and multiple-fault scenarios should be introduced together with incipient and abrupt types of faults. For each injected fault scenario, its degradation rate and resolution must also be fully configurable.
The following section provides a brief background on PHM data generation approaches by discussing examples in the literature where run-to-failure, seeded faults and fault emulation were used to support development of diagnostic and prognostic algorithms. Investigating, understanding and quantifying component degradation and failure is an integral part of component design and manufacturing (Limon et al., 2017;Childs, 2019). Commercially available products and components are designed, manufactured and machined against design standards for safety over their expected lifespan (Childs, 2019). The design itself is typically validated through actual component testing, to ensure compliance with the safety standards and legal liabilities, as well as to enable the manufacturer to confidently provide an expected lifespan for his products and of course warranty periods and recommended service intervals (Sander & Wang, 2000).
From this scope, the most recognized and accepted methods over the past decades in the manufacturing industry are the various accelerated testing (AT) methods. The various AT methods developed, aim to shorten product design and development lifecycle as well as to validate component reliability. The main reason that led to the development of the various AT methods is the fact that for modern high quality and reliable components or systems, any observable degradation is very difficult or even impossible to appear in a very short time period when the component or the system is subject to its nominal operating conditions. The main feature of the various AT methods developed is the induction of component degradation and failures by subjecting the component under test to increased duty cycles and accelerated "Run to Failure" schemes without at the same time creating any new or unaccounted failure modes. The most common AT methods and their features are collected and presented in a compact and convenient way in the study of Limon et al., (2017). The conclusion is that AT testing is a typical procedure carried out by OEMs (Original Equipment Manufacturers) and although it provides very reliable and accurate results and insights regarding component failure and degradation modes, is a very expensive procedure and requires the design and development of specialty machinery and testing facilities that will enable the efficient conduction of these tests. Furthermore, the fact that AT is a procedure funded and supported by OEMs makes it even more difficult to find experimental datasets and detailed information available in the public domain. Thus, from the PHM research perspective, apart from investigating degradation and failures of high or low-speed rotating machinery, especially bearings and gears, this method is not the common one for investigating component degradation and their effects at the system level (Wang et al., 2018).
Another method widely used for component degradation investigation in PHM studies and research is the seeded faults approach. This approach requires the failure modes under investigation to be known beforehand and based upon this knowledge to machine the components under test in such a way to represent a certain level of degradation. This level of degradation can be gradually increased by applying the relevant machining on the component under test. This approach provides accurate information only at discrete degradation levels. In order to map the full degradation spectrum, the component under test must be subjected to various machining steps to accurately capture the degradation phenomenon (Niculita et al., 2014). This approach has found wide application to both high and low-speed rotating machinery, power drives and power trains, as well as to component structural integrity investigation. The majority of the investigated high-quality research available in the public domain incorporates the seeded-fault approach for benchmarking PHM applications for both military and civilian aerospace and ground transportation vehicles (Hess & Hardman, 2002;Orsagh et al., 2004;Watson et al., 2007;Roemer at al., 2007;). The seeded faults approach is also an expensive process, as it requires machining of real components, and from the available literature it is obvious that this kind of research is typically backed up from joint industry-academic research funding schemes, or conducted in conjunction with state funded research facilities and institutions (Hess et al., 2000;Hess & Hardman, 2002;Hess et al., 2003).
The third method for investigating component degradation is the emulation of faults. This technique enables the emulation of the degradation effects of some components and their impact at the system level; in this particular application, an example of this technique is the utilization of a direct-acting control valve to emulate the degradation of a filter in a fluid system by gradually closing the valve (to replicate the clogging phenomenon). This method has found application to process-oriented systems, like fluid systems (Niculita et al. 2014;Skaf et al., 2015;Lin et al., 2017). For the emulated faults, the main limitation is the utilization of an actuator that is capable to emulate the effects of the desired component under test. While this is feasible for the case of filters and leakages in a fluid system, this method is not easily applicable to high or low-speed rotating machinery. However, due to the compatibility of this method with the degradation effects and fault scenarios investigated in this physical testbed, this approach was selected for this work. Moreover, this method, from the scope of a diagnostics testbed is realizable at an acceptable cost, as the testbed can be built from commercial off-the-self equipment, thus, making it an attractive choice to a wider audience.
The paper is organized as follows: Section 2 presents a literature review on diagnostics and prognostics testbeds, while Section 3 introduces the testbed and expands on the process of requirements capture for the actual testbed and its PHM capability. Section 4 discusses the data integrity elements, standards and techniques employed throughout the construction of the rig to guarantee the integrity of the generated datasets. Section 5 gathers the contributions to the topic and the concluding remarks of this paper.

DIAGNOSTICS & PROGNOSTICS TESTBEDS -A LITERATURE REVIEW
Various testbeds have previously been developed for benchmarking diagnostics and PHM applications and for the investigation of the degradation of equipment. The information available in the public domain for some of the previously developed testbeds is reviewed below and these experimental testbeds are benchmarked against a set of criteria. The intention of this literature review of the presented testbeds is not to compare the PHM applications and solutions developed or asses their quality and performance, but rather to investigate the extent and the detail of the information provided on the actual design and development of the testbeds themselves. Table 1 contains a condensed and comprehensive comparison between the reviewed testbeds with the aforementioned intention in mind.
The review of the various testbeds captured in Table 1 shows that the primary focus was given from the researchers, naturally, to the presentation of the PHM methodologies and on the results demonstrating the effectiveness of the methodology and not on the specific design considerations on the testbeds development themselves.
The articulation of clear guidelines on requirements and data integrity considerations for bespoke diagnostics and prognostics testbeds is not covered in the literature and it is this particular gap that this paper is trying to address. The adequacy of insights and elaboration on the following important topics and criteria related to experimental testbeds design are addressed in this benchmarking exercise and they are condensed in Table 1 (marked with *).
Due to the space constraints, only the benchmarking and comparison criteria captured in Columns 6, 7, 8 and 10 will be covered below. A more detailed discussion on the benchmarking exercise covering the existent PHM testbeds available in the literature was discussed by Bardakis (2019). We argue that PHM testbeds should be designed like any other industrial system or application and the following paragraphs will discuss the importance of standards and technical documentation when designing experimental apparatus capable of supporting PHM design and development studies.
The design of systems and products against the available standards applying to various regions in the world, with a main principal focus on safety and a secondary focus on performance and efficiency is a regulatory requirement. The different applicable standards depend on the nature and type of system or product (Childs, 2019). The importance and benefits of design against standards, apart from regulatory compliance, are many (Sands & Slaugenhaupt, 2017), with the most notable being: • Standards ensure safety and minimize legal liability. • Standards already embed best design practices. • Standards ensure system modularity, sustainability and maintainability.
• Standards provide design traceability.
Apart from some testbeds used for benchmarking PHM solutions targeting military applications, where there is a reference to requirements for the final system only, but not for the testbed design, (Brotherton et al., 2003;Delaney et al., 2009), the use of standards for the design process of such testbeds and for achieving data-integrity is not mentioned in the reviewed literature.
Data acquisition and analysis is one of the fundamental and most crucial aspects of any proper PHM application, as their results, and therefore any decision making relies on the processing of the acquired data Madhikermi, 2017;DNV GL, 2018). For a PHM application, data integrity reflects on the raw sensor data itself, in terms of accuracy, timestamping and timeliness, accessibility, repeatability, completeness and context (Kwon et al., 2014). Postprocessing of low-quality raw data greatly affects any calculations and results downstream, resulting in poor results and poor decision-making in terms of maintenance actions (Madhikermi, 2017). Achieving data integrity is intertwined with identifying, understanding and dealing with sources of data uncertainty, while at the same time ensuring the repeatability and the reproducibility of the test results. Identifying and dealing with uncertainties is a fundamental concept of measurement theory and measurement systems (Bentley, 2005). In the majority of the reviewed literature, data integrity is assumed and specific measures and strategies to deal with sources of uncertainty that affect data integrity are not mentioned.
The major source of uncertainty affecting data integrity highlighted by the PHM literature is signal noise. Apart from this observation, no other information or insights are provided regarding the sources of noise. Nor is any mitigation strategy employed to deal with sources of noise and interference at the system design level by following standards and recommended practices (RPs). An exception to this statement is the NASA ADAPT testbed (Poll et al., 2007;Feiyi & Jinsong, 2015). A very comprehensive package of documentation, drawings and schematics is provided by the NASA Ames Research Centre, where the application of standards is indirectly referenced, as it is captured in its accompanying drawings. Furthermore, the various noise mitigation strategies that contribute to the achievement of data integrity are indirectly referenced in accompanying documentation and drawings for the ADAPT.

PHM TESTBED REQUIREMENTS & SPECIFICATIONS
Key aspects that govern system design are its requirements and specifications. The introduction and proposal of tailored requirements that govern the design of diagnostics testbeds was a fundamental part of this exercise. The essential requirements for the design and implementation of diagnostics testbeds are condensed in Table 2, along with brief information on how these requirements were practically achieved in the case of the fuel system testbed.
Along with the individual requirements captured in Table 2 under specific categories, we argue that a map of the relationships between these requirements should be also produced. The main reason is that there is a high degree of direct and indirect interaction and dependecies between the requirements themselves. The map associated to the PHM, mechanical, electrical, software and application requirements considered throughout the design process of this particular testbed is pictured in Figure 1. For ease of understanding and to provide the relevant perspective, the flow-down of the PHM Requirement No1 (PHMR1) into other relevant requirements is highlighted in red in Figure 1. In order to communicate the rationale and the thought process for the introduction of the requirements and their translation and application in the actual fuel system testbed, the highlighted relationships of PHMR1 in Figure 1 will be described in more detail. Prior to delving into this analysis, it is convenient to briefly introduce first the topology of the designed fuel system testbed in Figure 2, as well as some of its essential functions. Figure 2. A simplified P&ID of the fuel system testbed A simplified P&ID (Process & Instrumentation Drawing) of the developed fuel system testbed is shown in Figure 2, with the testbed consisting of the following components: • A main supply tank.
• An external gear pump.
• An induction motor driving the pump.
• A variable speed drive (VSD) controlling the rotational speed of the motor, and thus the rotational speed of the pump.
• A system control panel.
• A print mark contrast laser sensor (for measuring the pump rotational speed) • PVC plastic tubing.
• A finger valve (to isolate the tank from the system).
The fuel system consists of a main fuel line/path and two leakage lines/paths (marked in red in Figure 2) to collect the injected leakages. The various DPVs throughout the system are used to create the various faulty scenarios considered for this testbed; the control action of the DPVs representing the degradation of the respective actual components. The DPV1 is used to emulate the clogging of the suction filter (DPV1 initially fully opened for healthy condition). The DPV2 is used to emulate a leakage right at the discharge side of the external gear pump (DPV2 being initially fully closed for the healthy scenario). The DPV3 is used to emulate the degradation of the by-pass shut-off valve SHV (stuck/degraded valve), as a blockage downstream of the pump. The DPV4 is used to emulate the degradation/clogging of a fuel nozzle, as a blockage further downstream of the pump. The DPV5 is used to emulate a pipe leakage further downstream from the pump. Thus, with respect to Figure 2, a complete list of the faults that can be accommodated by the PHM testbed are listed in Table 3 with their respective fault code and description. The actual capacity of fault injection scenarios is reconfigurable and expandable.
With reference to Figure 1 and Table 2, PHMR1 is an essential PHM requirements, as it states that "the testbed shall provide the ability to investigate a sufficient number of different fault scenarios". In order to maximize the PHM capability of the testbed, its technological capacity in terms of equipment, hardware and software must be fully exploited. The fulfilment of PHMR1 requires the testbed to be instrumented by a sufficient number of sensors able to measure adequate parameters that enable characterization of the components degradation and support the relevant PHM analysis and the investigation of the effects of the various fault scenarios at the system level; hence, the link between PHMR1 and PHMR2. Table   2.  The various sensors utilized in the fuel system testbed were mentioned above. The testbed shall provide the ability to easily interchange sensors of different types and technologies, in order to enhance and extend the PHM investigations accounting for sensors' measurement overall performance. This is achieved through the smart design of the system's control panel, the proper specification of the DAQ and control equipment and the fully configurable system. Thus, the relation between PHMR1, PHMR5 and ELECR2. Furthermore, a proper industrial testbed setup must allow for adequate expansion in terms of its supported Inputs/Outputs (I/O) and measurement equipment. This is achieved through the proper electrical and layout design of the system's control panel; hence the highlighted relation between PHMR1, PHMR4 and ELECR3.

List
In order to enable an application-neutral interoperability between the developed testbed and the various software applications developed by prospect researchers that will use the testbed for PHM research, the data acquisition and control functions must be made available and accessible. This is achieved through the provision of an open-standard API, in the form of a Modbus Registers table accessible either through Modbus TCP or Serial protocol. Moreover, as the testbed's control system architecture is fully networked and built upon an Ethernet backbone, bidirectional interaction with the testbed is applicable remotely either through the Local Area Network (LAN) or through the WAN (Wide Area Network) via Internet remote access through a dedicated secure VPN (Virtual Private Network) tunnel. This describes the relation between PHMR1, PHMR3 and APPR4.
One of the factors that define overall data integrity is the achievement of measurement repeatability and test results reproducibility. The testbed's fuel system setup is a fluid, hydraulic system, comprising the relevant piping and hydraulic equipment (e.g. gear pump, tee-junctions, valves, etc.) configuration. As such, the physical layout of the fuel system defines its system curve (according to the applicable pressure losses) for the various pump speeds. Thus, in order to achieve repeatability and reproducibility the system's elements (piping, instrumentation and equipment) shall be securely fixed in order to achieve fixed hydraulic system curves. Even a slight displacement or elevation of a system piping section results in significant variations in the system's hydraulic curves and thus greatly affects repeatability and reproducibility. This describes the relation between PHMR1, APPR5 and MECHR1.
Another important consideration is the execution of experiments without interruption over long periods of time, even at full nominal load, in a safe manner, with or without local supervision. The testbed is designed to operate safely in a 24/7, completely unattended UMS (Unmanned Machinery Space). Dedicated UMS tests were performed in order to verify and validate the UMS capability of the testbed. The parameters of such a test are presented in Table 4. All the possible cases of failures and abnormal operating conditions are considered and embedded in the safety interlocks and Emergency Shut Down (ESD) logic programmed in the system's PLC controller. In the unlikely event that such a situation arises, the programmed safety ESD logic will kick in and shutdown the system, following a preprogrammed ESD procedure and steps. The testbed's instrumentation and the designed self-diagnostic functions in the controller are continuously monitored by the safety and ESD logic. This justifies the link between the requirements named: PHMR1, APPR3, SWR1 and SWR5.
At this point, it is important to mention that, as the testbed is to be treated and designed as a real industrial system, its design should be governed by the applicable industrial standards for safety, performance and reliability. The main objective of the application of standards is the safety of the operators, followed by the protection of machinery. Safety precedes any system performance and efficiency considerations, either related to data integrity, measurement performance, etc. Thus, the testbed is designed following applicable industrial standards. These standards determine the specifics of the safety and ESD logic to be programed in the controller, as well as any safety design aspect of the testbed (e.g. protection against electrical hazards, EMC, etc.). As the instrumentation (employed for investigating the various PHM related fault scenarios) comprises a fundamental part of the electrical application, its implementation in the testbed design is governed by the applicable standards. Moreover, the application of industrial standards, rules and recommended practices, apart from safety, boosts data integrity through the augmentation of measurement performance by mitigating signal noise (EMC, control of harmonics, grounding and equipotential bonding, etc.). Thus, the relation between PHMR1, APPR2 and ELECR1. A list of the main industrial standards, rules and recommended practices used in the design and development of the fuel system testbed is presented in Table 5.
Furthermore, with the exception of few high-end PHM testbeds, the importance and application of standards with regards to safety, operation, traceability of the testbed design, life-cycle support and ultimately data integrity are not highlighted and documented in the PHM-related literature. The authors deemed it important to follow industry applicable standards on the design and development of all the aspects of the new testbed (Electrical, Safety, Software, HMI, etc.), which combined with the production of the engineering drawings and documentation, form another key differentiating point compared with the investigated literature and information in the public domain.
From the context of a PHM application based on the emulated faults methodology, sensor measurement alone may not provide the necessary insight for the postprocessing of the data and the extraction of informed results. In the designed testbed, apart from the sensor measurements, all the system operational parameters are logged in the data log file and time stamped offering a contextualized system level wide view for various levels of severity of the degradation phenomenon. Thus, the relation between PHMR1, PHMR6 and SWR2.
All the aforementioned specifics of the various design requirements are of no value if they are not documented. The designed fuel system is accompanied by a comprehensive set of technical documentation and drawings (electrical, mechanical and software). Thus, the relation between PHMR1 and APPR1. Figure 4 provides a snapshot of the developed system drawings and documentation (available in Bardakis, 2019), where all the design details are included, as per the relevant standards, rules and recommended industrial best practice, along with the actual developed fuel system testbed, which was the result of transforming the drawings and conceptual design to reality. The thought process behind the design and development of this particular application showcases the importance of the requirements for the design of diagnostic PHM testbeds. This is highlighted in the system specifications phase of design and development process depicted in the flowchart in Figure  5, which shows the detailed steps and procedures followed by the authors for the design and development of the new testbed. As the requirements are set at the initial design stages of a system, they govern and dictate all the design aspects and details of the system, and in our case the diagnostics testbed.
From a procedural point of view, the groups of the individual requirements described in Table 2 are applicable for a wide range of diagnostics and PHM testbeds, and the majority of them can be considered as universal requirements. The PHM requirements are set first, followed by the application requirements. These two sets of requirements determine and set the relevant electrical, mechanical and software requirements, which the overall testbed design will be based upon. Thus, from the requirements listed in Table 2, we propose the ones having their number marked with (*) to be considered for universal application for diagnostic and prognostics testbeds. The rest of them are testbed-specific requirements, they can be shaped to fit the purpose accordingly, either reducing or augmenting this lust, and are typically dictated by the universal ones.

DATA INTEGRITY
Data integrity is a critical aspect to the design, implementation and use of any system which generates, stores, processes, or retrieves data. In the context of experimental testbeds, data integrity is the assurance of the accuracy and consistency of data. Throughout the design of the proposed hydraulic system testbed, we have addressed several factors which are acknowledged to impact upon data integrity. Although a good number of papers in the literature highlight the importance of the measurement uncertainty in the context of PHM applications (Watson et al., 2007;Sankararaman & Goebel, 2015), this paper will cover several other factors impacting the integrity of the data generated by this testbed, as well as sources and hardware/software limitations that affect the overall measurement uncertainty and performance. For each of these factors, we will discuss the measures put in place to address and control them as the system was designed and then built. A mapping between the factors and the phase within the design and development process will be also provided. We believe this mapping will help future PHM research to better design and develop bespoke PHM testbeds. From a system-level perspective, for the proposed system, we pursued data integrity by identifying and mitigating the sources and factors that affect it. These sources fall into two main categories: measurement uncertainty and measurement performance. Measurement uncertainty depends on random and systematic errors affecting the quality and confidence of the data generated from the system, whereas measurement performance depends on system-level and application-specific factors that affect the whole process of acquiring, logging and processing data.
The following sections elaborate on these factors and also highlight the measures taken throughout the construction of the rig in order to mitigate their effects. When specific measures were not implemented due to time and/or budget limitations, they are presented as recommendations for potential upgrades or elements to be considered for future diagnostics and PHM testbeds.

Measurement uncertainty -Random errors
As identified in the reviewed literature, the major random error affecting measurement uncertainty is signal noise due to interference. In the testbed design, this issue is dealt with, and greatly mitigated, through a series of measures related to proper engineering design against the applicable standards and recommended practices, leading to a minimum of a six times of noise reduction, compared to the scenario when such measures are not implemented. These measures were considered part of the design process and they refer to the use of twisted, shielded and properly grounded armored instrumentation cable, the guiding of power and motor cables through flexible, grounded steel conduits, cables separation and routing according to their class, RF (Radio Frequency) bonding, and the use of EMC compliant equipment like the PLC (Programmable Logic Controller, the power supply and the network equipment. These measures were identified and considered within the SS2, SS3, SS4 and SS5 phases, introduced in the design during the DED1 to DED3 phases and implemented throughout the SCon1 to SCon4. To illustrate the effects of noise and interference, we have performed a noise study, comparing the induction of parasitic voltages due to the operation of the VSD controlling the pump motor (which is acknowledged to be a major source of interference and power line pollution), for the worst case scenario (signal cable running in parallel right next to the motor cable). A snapshot of the measurements taken for this noise study with a DSO5012 digital oscilloscope is included in Figure 6. The application of the interference mitigation measures has ensured stable and consistent noise mitigation throughout the whole operating frequency range of the VSD. Case 1 to Case 5 capture instances where the previously mentioned standards were not applied, while Case 6 demonstrates the mitigation of the effects of the interference generated by the VSD.
A typical and highly accepted approach to any data acquisition and data processing application, especially when operating in a noisy environment, is the implementation of either hardware filters (low/high pass, etc.) or software filters (moving average, etc.), in order to smooth the data and reject as much as possible from the noisy part of the signal. However, the overall performance of the filters themselves depend on the quality of the raw signal data itself, thus, the overall output of such filters is affected by the noise content of the signal and its susceptibility to interference. We support the approach that the signal noise and interference must be dealt with and mitigated through proper electrical and mechanical engineering design first, prior implementing signal filtration, especially software filters. Furthermore, there are PHM applications, whose results and performance depend on the quality of the raw, unfiltered data, such as machine vibration data and other high-frequency measurements or unique incipient fault signatures. In such cases, the useful information may be hidden and masked in the noise content of the signal. To demonstrate the overall data integrity performance achieved with the testbed's design, all the data we present in this paper are raw data.

Measurement uncertainty -Systematic errors
Several factors contributing to the systematic errors identified for this system will be discussed in the next section.

Meter Loading
The authors quantified the meter loading error for each analogue sensor used in this setup. All the analogue sensors provide a voltage output in the range of 0-5 VDC. The quantification results are presented in Table 6 below. The methodology to calculate meter loading error is documented in the literature (Bentley, 2005). The factors related to meter loading were considered during the DED1 phase of the design process during the instrumentation specification based on their characteristics (accuracy, resolution, drift, mechanical design, etc.). In the case of this testbed, the meter loading calculations were carried out as part of the instrumentation verification process, as existing sensors from the laboratory's inventory were utilized for the proposed testbed development. The calculated values lie well within the datasheet specifications of the system components (sensors and PLC). Also, from the systematic errors affecting measurement uncertainty, meter loading is one of the few that can be accurately quantified.

Operating Temperature Effects
All the measurements and the experiments were conducted under a steady ambient environmental temperature of 25±1 o C within the laboratory and only when, the water in the testbed's supply tank has reached thermal equilibrium with the ambient environment. In addition to the above condition, all the measurements took place when the system was already in operation for at least one hour, in order for the critical parts (like the motor, the pump and the valves) to reach their nominal operating temperature, as it would be the case in a continuous industrial operating scenario. This nominal thermal operating condition for the components was validated with consecutive thermal imaging measurements, using a FLIR (Forward Looking InfraRed) thermal imaging camera (Figure 7).
During the conduction of the experiments, it was observed that, if tests were conducted when the tank was filled with fresh tap water, at a much lower temperature than the ambient (i.e. water temperature at 11 o C and ambient temperature at 25 o C), the driving of the water through the pump actually cooled down the pump casing, which in turn, cooled down the motor. This typically led to a slightly better overall motor efficiency. To avoid such inconsistencies generated by the temperature effects, the data acquisition and logging campaigns were carried out under full thermal equilibrium.

Sensor Offset and Sensor Calibration
Sensor drift (or long-term stability) and sensor offset are two other major sources of measurement uncertainty in terms of systematic errors. These were identified by the authors throughout the design and commissioning of the proposed PHM testbed. These sources and their maximum allowable and expected effect per year on the sensor measurement were considered in DED2. In order to gain some insight on the status of the existing pressure transmitters in terms of drift and offset, a simple and effective test was performed. All the pressure transmitters were subjected to the same water column (500 mm of water manometric height) and the relevant pressure measurements were recorded. The findings indeed displayed offset from the expected value for all the sensors, as well as differences between the different sensors as well. This indicates that all the sensors display some level of offset, and this offset and drift is different even for sensors of the same model and type. According to manufacturer's datasheet, the pressure transmitters P3, P4 and P5 have a built-in offset, as they are non-true zero output sensors. The determination of the sensor offset, and drift is a matter of proper sensor calibration. The calibration of the testbed's sensors should be carried out at the start and following that, at regular intervals to guarantee the integrity of the generated data capturing healthy and faulty conditions emulated/injected in the system.

Facility Power Supply Voltage Fluctuations
One of the factors that can affect measurement performance, is the voltage fluctuation of the AC (Alternating Current) mains power supply. The variability of the supply voltage can create slight disturbances to the actuating components. The proposed PHM testbed is powered from the facility mains power distribution system. During various power supply measurements performed by the authors, using a True RMS digital multi-meter to measure the AC mains power, a voltage fluctuation between 242 and 252 VAC was observed. This voltage fluctuation can slightly affect the VSD output voltage to the induction motor. For example, at the 100 % output of the VSD (50 Hz), the fluctuation of the AC mains power supply voltage is reflected to a phase-to-phase voltage fluctuation at the VSD motor terminals with values anywhere between 268 and 272 VAC. This voltage is higher than the supplied AC mains voltage at the input of the VSD due to its internal rectifying bridge and its DC (Direct Current) Bus. The voltage fluctuation at the motor terminals can have a slight effect on the actual pump speed. For example, for a fixed setpoint of 1480 RPM, the actual pump speed can fluctuate up to ±2 RPM from the desired setpoint. The impact of the power network fluctuations on the measurement uncertainty was considered during the following phases of the design and development process: SS3, SCom1, SCom2 and LTS1. The effects of the voltage fluctuation on the data generated by the PHM testbed can be mitigated through the utilization of a combination of an independent, calibrated and stabilized power supply, a highly efficient VSD and a calibrated motor.

Performance of the PLC's D/A converter and analog outputs wiring
The rotational speed of the pump motor is controlled via a VSD. In the implemented system's architecture, the speed reference command is provided to the VSD in the form of a 4...20 mA current control signal from the PLC. The pump speed setpoint set at the HMI is sent to the PLC and converted to a 4...20 mA signal through the PLC's Digital to Analog (D/A) converter. The respective analog output of the PLC is hardwired with the respective control terminals of the VSD.
The combined performance of the PLC's D/A converter along with the performance of the analog output wiring (in terms of noise susceptibility) have a direct effect on the actual speed reference command applied to the VSD. The most significant factor affected by the performance of the PLC's D/A is the offset from the desired setpoint as a byproduct of the digital to analog conversion process, whereby the signal noise can affect the fluctuation and oscillation of the control signal around the setpoint. In the case of the implementation in this testbed, the mitigation of signal noise and interference achieved, practically eliminated any fluctuation on the desired speed command. However, the performance of the D/A converter remains an issue. Ideally, this issue would be resolved by the utilization of a different technique to control the pump speed by using a VSD with embedded communication capability based on an industrial communication protocol (e.g. Modbus). In that case, the speed command would be sent directly from the PLC to the VSD digitally, through the communication link. Under these circumstances, the only factors that would affect the applied pump speed, would be the efficiency of the VSD as a power inverter itself primarily, and the facility's power supply voltage fluctuation, as already mentioned in the previous section.

Sampling time
The system architecture implemented in this testbed considered functional safety, control logic and data processing functionality. These features are embedded in the PLC code and are executed in hard real time, while all the high level supervisory, monitoring and control functionality, as well as the data logging are performed through a remote, networked PC (Personal Computer) workstation where the HMI software application runs. The Modbus TCP (Transmission Control Protocol) industrial protocol was implemented for data exchange between the HMI application on the PC workstation and the PLC in the control panel. As this is a networked application, and due to the fact that the Modbus TCP protocol is not a deterministic protocol, the minimum sampling time that can be achieved for the system heavily relies on the protocol's performance over the LAN that this system actually operates. There are two other factors that also affect the minimum sampling time achieved. These factors are the PLC scan cycle time and the PLC's A/D (Analogue to Digital) conversion time. The PLC scan cycle for the developed PLC program controlling the PHM testbed is constant and its value is 780 ns at a PLC CPU (Central Processing Unit) load of 44 %. The PLC A/D conversion time is also constant at 10ms; this was obtained from the equipment datasheet. The figures mentioned above were included in the overall calculation of the minimum sampling time. Thus, in order to determine the sampling time, two commercially available network analysis tools (WireShark and Colasoft Capsa) were utilized, to perform a full network breakdown analysis at the wire level of the LAN and acquire the actual time that the Modbus TCP packets are exchanged over the network. The Modbus TCP data exchanged over the network consists of 40 Modbus Holding Registers and 17 Modbus Coils (Figure 8). The minimum sampling time for the implemented system architecture in the PHM testbed was determined to be 200 ms. The only limitation posed by this sampling time is that it does not allow for investigation and analysis of high-frequency phenomena, like water hammer and equipment vibration, which are out of the scope of the scenarios to be addressed by this PHM testbed.

Initial results highlighting repeatability and reproducibility of the measurements
The evidence supporting the implementation of the measures discussed in the sections 4.1 to 4.3 addressing factors impacting the data integrity is presented below. They highlight the repeatability and reproducibility of the results with the system running at the healthy operating point, but they also cover five faults emulated on this testbed. We focused the initial part of the investigation on the pump performance curves. The data behind these curves is an excellent demonstration of the overall degree of data integrity achieved through the design of the testbed, as it highlights the reproducibility, repeatability and the concentration of the raw data acquired. The results of the pump performance tests conducted are summarized in Figure 9, where a total of 11 different tests conducted on different dates and times are presented. Figure 9. Repeatability and reproducibility of the gear pump's performance curves at 1140 RPM Moreover, for the healthy operating point of the system, the throughput flow and the pump pressure are presented in Figures 10 and 11 for a series of ten different tests carried out with the pump running at 1140 RPM. As the testbed emulates a fuel system, which is actually a fluid system, its healthy operating point is governed by the physical layout of the system's piping and components, when there are no any blockages or leakages in the system, meaning that none of the emulated faults FC1 to FC5 are injected in the system at this point. The reason we chose to present the data for the pump performance curve and the healthy operating point at a fixed pump speed of 1140 RPM is due to the fact that the manufacturer of the gear pump used in the testbed (Maker: Oberdorfer, Model: N999R external gear pump) provides only the performance curve at 1140 RPM. Furthermore, the tests at 1140 RPM provided the opportunity to assess and compare the actual performance characteristics of the pump against the manufacturer's data. This task is a part of a full characterization process of the whole system, which will be described in more detail in a future paper.
The output of the 11 tests displayed in Figure 9 highlight the validation and verification of the application, mechanical, electrical, software requirements. The N999R gear pump model has a built-in safety relief valve having its default setting fixed at 3.4 bar (50 PSIG). As the load on the pump increases, the volumetric flow rate drops with the increase in pump pressure up at value set by the relief valve. From that point onwards the action of the relief valve generates the behavior displayed on the right-hand side of the red threshold. This behavior is highly repeatable further underpinning the robustness of the system. The graphs in Figure 10 capture the raw data of the parameters involved in the characterization of the pump acquired with the system running under healthy conditions (HC). The raw data presented are from 10 different tests conducted throughout a period of 3 days, in order to highlight the repeatability of the measurements.
A key factor that explains the distributions highlighted in Figure 10 relates to the operation of the external gear pump. The operation of an external gear pump in a fluid system creates pulsating flow due to the rotation of its gears, resulting in flow and pressure ripples in the system. The particulars of these flow pulsations depend on the pump's geometrical characteristics (gear geometry, etc.) and the rotational speed of the pump. The flow pulsations are on the discharge side of the pump and for this reason the distribution of the pump pressure (pump head) is shaped by the distribution of the pump discharge pressure (Figure 10a and 10d). The effects of the pulsating flow, as well as the relevant impact on its measurement through instrumentation (flow and pressure sensors), is thoroughly investigated in the literature (Manring & Kasaragadda, 2003;Frosina et al., 2017;Zongbin et al., 2018). The parameters captured in Figure 10 (a) and (b) are further explained and contextualized with regards to the pump healthy operating point and their low variability is governed by the data integrity measures implemented on the system. Another representation of the data shown in Figure 10 is Figure 11, which also highlights the level of data integrity achieved. The concentration and the limited spread of the data is plotted as a compact dot on the pump flow vs. pump pressure graph. The magnifying glass on the graph zooms on the actual healthy operating point of the pump. Its shape and distribution are the effects of the pulsating flow generated by the gear pump, described above. Figure 11. Repeatability and reproducibility of the healthy operating point Figures 12 (b) to 12 (f) demonstrate the robustness of the proposed PHM testbed when five faulty conditions (FC1-FC5) were emulated with various degrees of severity, while Figure 12 (a) shows the healthy operating condition for comparison purposes. In any industrial application containing a pump, it is widely accepted to have the performance of such a component monitored using the pump suction and discharge pressure as well as the throughput flow delivered down the line. For this reason, in the following figures, these three parameters were used to characterize the repeatability and reproducibility of the emulated tests. Three different tests gathering the variability of these three parameters were plotted for the five faulty scenarios.
The scenario that will be further explained is the one where the absolute reproducibility of the results heavily depends on the physics and the fluid dynamics of the system under specific conditions. Figure 12 (b) depicts the clogging phenomenon of a filter placed at the suction side of the pump. In order to interpret this figure, knowledge and understanding of how the gear pump works is required. A vacuum is created at the suction side of the gear pump as the pump's gears unmesh. The liquid is driven into the gear pump solely due to the pressure difference between the atmospheric pressure at the top of the main supply tank and the vacuum at the suction side of the pump.
As the filter gets clogged over time, this vacuum becomes stronger and stronger due to the operation of the gear pump. The result is that even for a fairly high severity (even up to 50 %), the delivered flow does not change significantly, because the pressure difference between the atmospheric pressure at the top of the main supply tank and the vacuum created at the suction of the gear pump is enough to drive the liquid into the pump. This pressure difference is enough up to the point where absolute vacuum appears at the suction side of the gear pump. After this point, as the filter clogging increases, the pressure difference is not enough for sufficient liquid to be sucked into the pump. This is where the degradation rate becomes very fast and strong cavitation in the pump appears. The effects of the strong cavitation are reflected in the measurements and can be seen by observing the fluctuations in the pump discharge pressure P3 and pump flow, from the point in time where absolute vacuum in the pump suction side appears and onwards. Furthermore, an interesting effect can be observed as degradation severity becomes very high. It is a sudden burst of flow. What actually happens, is that at this point there is no more flow from the tank to the pump, as the filter clogging is so severe, causing huge pressure losses across the DPV1 control valve (used to emulate FC1). Thus, at this point, the magnitude of the pressure losses across the DPV1 valve is exactly the same as the pressure difference between the atmospheric pressure on top of the tank and the vacuum at the suction side of the pump. This is the reason that at this point there is no more flow from the tank to the pump. A question that one can naturally raise here is, if at this point there is no more flow from the tank to the pump, how the sudden burst of flow after this point is generated. The explanation to this phenomenon is that what actually happens after this point, is that the pump just flushes out, in this sudden burst, whatever liquid remains trapped in the line, and then the flow stops completely. This phenomenon, in the maritime industry, is known as "line stripping". Similar observations apply to FC2 (Figure 12 (c)) which is related again to the fluid dynamic specifics. The other three scenarios these abrupt behavior does not occur due to the location in the system where the faults are emulated and their respective fluid dynamics. These are phenomena that a PHM algorithm will have to deal with, but in the case of this testbed, we have achieved a high reproducibility and repeatability even for these particular cases, as their unique fault signatures are always the same and fully repeatable.
Concluding with our study on data integrity, we acknowledge that data integrity itself is a very wide topic. In this paper, we elaborated on how we dealt with issues that we believe are important to consider in order to improve data integrity, such as the mitigation of the key sources of measurement uncertainty, the improvement of the measurement performance (such as measurement repeatability and sampling time performance). However, there are other data integrity aspects for consideration, such as data storage, data security and data completeness. Although functions such as the above are handled by the control and data acquisition built for this application, further elaboration on them falls out of the scope of work of this paper, which focuses primarily on the data integrity and quality of the raw data generated.
At this point, it should be noted, for completeness, that the system dynamics phenomena described herein, as well as the whole hydraulic system that the testbed represents, can be easily modelled. Standard fluid mechanics and dynamics models, as referenced in textbooks, can be easily applied to this case. The testbed comprises of standard components and parts such as the gear pump, straight pipe segments and bends, valves and instruments that have well documented mathematical models defined for them. The authors, during the design and the operation of the testbed, were able to verify the system against standard fluid mechanics models. Moreover, the software and HMI application developed for the testbed enables observations of the system's operation against these physics-based models. However, we believe that further elaborating into the mathematical and modelling aspects of the system, falls out of the scope of work of this paper.

SUMMARY AND CONCLUSIONS
The goal of this paper was to articulate a structured process for the design and development of testbeds supporting PHM research, in order to serve as a starting point and quick reference guide for engineers and researchers. The contributions of this paper mainly relate to requirements and data integrity considerations for bespoke diagnostics testbeds. These contributions are listed below:  Figure 1). -A detailed breakdown of the factors affecting data integrity and the methods used to mitigate them throughout the design and development process. -A guidance through the thought process followed by the authors to tackle the design challenges and considerations of the developed testbed, aiming to raise awareness of these design challenges.
This paper described the development of a diagnostic testbed capable of generating high-quality data suitable for testing algorithms underpinning incipient fault detection, system performance deterioration, fault isolation and damage estimation. The design and development of the PHM testbed was carried out with considerations on system specifications, detailed engineering design, system construction, workshop tests, system commissioning and long-term support planning. Several approaches adopted by the PHM community for generating data characterizing degradation were discussed. The emulation approach was employed for the proposed testbed and two large topics, we recognized as being instrumental throughout the construction of the diagnostics testbed, are discussed in detail: testbed design requirements and data integrity.
A clear mapping of requirements was carried out at the start of the system specifications phase and a discussion on how the application, mechanical, electrical and software requirements were dictated/derived from the main PHM requirements was included. The paper also highlighted the flow-down of one specific testbed PHM requirement (note to the reader: we labelled PHM requirements, the specifications the testbed needs to meet in order to generate high-quality data suitable for benchmarking PHM applications; this is not to be confused with the PHM performance requirements of a typical PHM capability).
The testbed provides, in a cost-effective manner, full degradation curves in a highly repeatable and reproducible manner and not just specific, distinct degradation levels as offered by the testbeds employing the seeded faults approach.
The evidence of the data integrity achieved for the system running under healthy but more importantly under faulty conditions (for the full degradation profile of five distinct faults) is presented in the last section and it demonstrates excellent repeatability and reproducibility even for severity levels and unique characteristics of some faults highly dependent on the fluid dynamics (see discussion on the clogged suction filter scenario at the end of section 4).
A limitation of this specific testbed, due to its system architecture, is the minimum sampling time. It is fully suitable for capturing the process dynamics and the signature of the faults considered under the investigation, however, it is not suitable for capturing and investigating high-frequency phenomena like water hammer. At this point, the system lacks electric and power quality measurements; such measurements can offer detailed insights on the motor and pump specific faults. Considering the approach taken for the system design, this feature can be easily integrated at a later stage.
A further direction for future work is to consider mechanisms to correlate effects of degradation curves of real components with the effects generated by a closing/opening of the DPVs used for emulation of such degradation profiles on the testbed. Currently, prognostics data are obtained by running to failure relatively low-cost components with the aim that the developed algorithms are scalable/transferable to other similar components. Very often, operating conditions, maintenance regimes, user input and various other factors of stress, completely change the shape of the degradation profile. The effectiveness of scaling up diagnostic and prognostic algorithms developed using emulated degradation profiles to real degradation of given components will be further investigated.

Ioannis Bardakis is an EI&C Systems Engineer at SELMA (Ship Electric Marine Automation). He has a MSc in Applied
Instrumentation and Control from the Glasgow Caledonian University and a Diploma in Mechanical Engineering from the National Technical University of Athens. He has over ten years of experience designing and developing industrial electrical, automation and control systems, mainly for the maritime and shipbuilding industries. His current research and engineering interests include the integration of condition based monitoring and predictive maintenance capabilities, from a performance-oriented approach in the systems he designs. He is a chartered engineer and member of the Institution of Mechanical Engineers.