Designing for Human-Centred Decision Support Systems in PHM
Prognostics and health management (PHM) represents a paradigm shift from legacy condition based maintenance (CBM) frameworks by expanding the potentials to accurately and robustly detect and diagnose incipient system faults. The ultimate goal of PHM is reliably predicting system failure times to allow for efficient maintenance scheduling either autonomously or by human decision makers (DM). In many industrial settings today the output from PHM systems constitutes a decision support system (DSS) used to aid DM, as entirely autonomous systems have not seen widespread industrial integration. However, there is relatively little support for engineers designing PHM systems in terms of human factors and how to provide the information in a way that actively supports human decision-making and this gap may result in limited use of PHM system by maintainers. The reliability of the information presented is a critical factor in the user acceptance and trust in a system. As a first step in developing such guidance, this paper reviews the implementation of other DSS and presents a design framework whereby PHM reliability levels are mapped against a suggested level of human input to the decision making process regarding required maintenance. The aim is to provide engineers with a guide to the level to which they should consider human factors and the presentation of information in the design of their PHM system. Fundamental to the suggested paradigm is that the uncertainties within a PHM system can be quantified, and as uncertainty increases, the requirement for DM to access additional information not explicitly tied to the PHM output increases. This information can form both explicit and tacit knowledge of a system or indeed industrial contexts surrounding decision implications, such as acceptable maintenance intervention windows in busy production schedules. As the complexity of a system or component being monitored is likely to affect the uncertainty within the PHM system associated with it, we are considering the overall cumulative uncertainty of a model output as the metric through which the required level of human input can be inferred. Coupled to this is the fact that increased model uncertainty is a causal factor in distrust and potential non-use of the model in industrial applications. It is the authors’ belief therefore that designing for increased human-model interaction concurrent with increasing model uncertainty may lead to a better engagement with PHM decision support capabilities, thereby offering the full advantages that PHM has to offer. The framework presented in this paper is an initial step towards facilitating the design of more usable and useful PHM systems.
How to Cite
uncertainty management, decision support, prognostics and health management (PHM), human factors, decision making with uncertainty
Australian Government Civil Aviation Authority. (2013). Safety Behaviours: Human Factors for Engineers Resource Guide.
Aven, T., Baraldi, P., Flage, R., & Zio, E. (2014). Uncertainty in Risk Assessment. (T. Aven, P. Baraldi, R. Flage, & E. Zio, Eds.). Chichester, United Kingdom: John Wiley & Sons, Ltd. doi:10.1002/9781118763032
Balfe, N. (2010). Appropriate Automation of Rail Signalling Systems: A Human Factors Study. University of Nottingham.
Baraldi, P., Di Maio, F., Genini, D., & Zio, E. (2013). A Fuzzy Similarity Based Method for Signal Reconstruction during Plant Transients. In Prognostics and System Health Management Conference PHM-2013 (pp. 889–894). doi:10.3303/CET1333149
Baraldi, P., Di Maio, F., Pappaglione, L., Zio, E., & Seraoui, R. (2012). Condition monitoring of electrical power plant components during operational transients. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 226(6), 568–583. doi:10.1177/1748006X12463502
Baraldi, P., Di Maio, F., Rigamonti, M., & Zio, E. (2013). Transients Analysis of a Nuclear Power Plant Component for Fault Diagnosis. Chemical Engineering Transactions, 33, 895–900. doi:10.3303/CET1333150
Bechhoefer, E., & Morton, B. (2012). Condition monitoring architecture: To reduce total cost of ownership. In 2012 IEEE Conference on Prognostics and Health Management (pp. 1–9). Denver, CO: IEEE. doi:10.1109/ICPHM.2012.6299509
Ben-Daya, M., Duffuaa, S. O., Raouf, A., Knezevic, J., & Ait-Kadi, D. (2009). Handbook of Maintenance Management and Engineering (1st ed., p. 768). London: Springer. Retrieved from http://books.google.ie/books?id=WE2M8YAD7jQC
Berner, E. S. (Ed.). (2007). Clinical Decision Support Systems: Theory and Practice (2nd ed., p. 269). New York, NY: Springer New York. doi:10.1007/978-0-387-38319-4
Billings, C. E. (1991). Human-Centered Aircraft Automation : A Concept and Guidelines. NASA Technical Memorandum 103885.
Bohn, H., Bobek, A., & Golatowski, F. (2006). SIRENA - Service Infrastructure for Real-time Embedded Networked Devices : A service oriented framework for different domains. In Proceedings of the International Conference on Networking, International Conference on Systems and International Conference on Mobile Communications and Learning Technologies (ICNICONSMCL’06) (p. 43). Washington, DC, USA.
Butler, M. (2013). Predictive Analytics in Business: Strategy, Methods, Technology (p. 28). Retrieved from http://butleranalytics.com/wp-content/uploads/2013/08/Predictive-Analytics.pdf
Crocoll, W. M., & Coury, B. G. (1990). Status or Recommendation: Selecting the Type of Information for Decision Aiding. In Proceedings of the 34th Annual Meeting of the Human Factors & Ergonomic Society (pp. 1524–1528). Santa Monica, CA: Human Factors and Ergonomics Society.
Deugd, S. De, Carroll, R., Kelly, K. E., Millett, B., & Ricker, J. (2006). SODA : Service-Oriented Device Architecture. Pervasive Computing, IEEE, 5(3), 94–96.
De-Vries, P., Midden, C., & Bouwhuis, D. (2003). The effects of errors on system trust, self-confidence, and the allocation of control in route planning. International Journal of Human-Computer Studies, 58(6), 719–735.
Ding, J., Hines, J. W., & Rasmussen, R. (2003). Independent component analysis for redundant sensor validation. In Proceedings of the 2003 Maintenance and Reliability Conference (MARCON 2003). Knoxville, TN.
Dreiseitl, S., & Binder, M. (2005). Do physicians value decision support? A look at the effect of decision support systems on physician opinion. Artificial Intelligence in Medicine, 33(1), 25–30. doi:10.1016/j.artmed.2004.07.007
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003). The role of trust in automation reliance. International Journal of Human-Computer Studies, 58(6), 697–718.
Engel, S. J., Gilmartin, B. J., Bongort, K., & Hess, A. (2000). Prognostics, the Real Issues Involved with Predicting Life Remaining. In Proceedings of IEEE Aerospace Conference (pp. 457–469).
Evans, P. C., & Annunziata, M. (2012). Industrial Internet : Pushing the Boundaries of Minds and Machines (p. 37).
Garcıa-Alvarez, D. (2009). Fault detection using principal component analysis (PCA) in a wastewater treatment plant (WWTP). In In Proceedings of the International Student’s Scientific Conference.
German Federal Ministry of Education and Research. (2013). Zukunftsbild „Industrie 4.0“ Hightech-Strategie (p. 36).
Gertler, J. (1998). Fault Detection and Diagnosis in Engineering Systems (p. 504). New York: Marcel Dekker Inc.
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association : JAMIA, 19(1), 121–7. doi:10.1136/amiajnl-2011-000089
Guerlain, S. A., Smith, P. J., Obradovich, J. H., Rudmann, S., Strohm, P., Smith, J. W., … Sachs, . (1999). Interactive Critiquing as a Form of Decision Support: An Empirical Evaluation. Human Factors: The Journal of the Human Factors and Ergonomics Society, 41(1), 72–89. doi:10.1518/001872099779577363
Hines, J. W., & Garvey, D. R. (2006). Development and Application of Fault Detectability Performance Metrics for Instrument Calibration Verification and Anomaly Detection. Journal of Pattern Recognition Research, 1, 2–15.
International Civil Aviation Organization (ICAO). (2003). Human Factors Guidelines for Aircraft Maintenance Manual. Montréal, Quebec.
International Ergonomics Association. (2000). What is ergonomics? Retrieved from http://www.iea.cc/browse.php?contID=what_is_ergonomics#sthash.gpOFBKYx.dpuf
International Standards Organisation. (2010). Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems. ISO Standard 9241.
Jain, A. ., Duin, R. P. W., & Mao, J. (2000). Statistical Pattern Recognition : A Review. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 22(1), 4–37.
Jardine, A. K. S., Lin, D., & Banjevic, D. (2006). A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical Systems and Signal Processing, 20(7), 1483–1510. doi:10.1016/j.ymssp.2005.09.012
Kalgren, P., Byington, C., Roemer, M., & Watson, M. (2006). Defining PHM, A Lexical Evolution of Maintenance and Logistics. In 2006 IEEE Autotestcon (pp. 353–358). IEEE. doi:10.1109/AUTEST.2006.283685
Karnouskos, S., Colombo, A. W., Jammes, F., Delsing, J., & Bangemann, T. (2010). Towards an architecture for service-oriented process monitoring and control. In IECON 2010 - 36th Annual Conference on IEEE Industrial Electronics Society (pp. 1385–1391). Glendale, AZ: Ieee. doi:10.1109/IECON.2010.5675482
Ketteler, G. (1999). Analysis of Requirements for Monitoring Systems. In H. Van Brussel & P. Valckenaers (Eds.), Proceedings of the Second International Workshop on Intelligent Manufacturing Systems (pp. 721–725). Leuven, Belgium: Katholieke Universiteit Leuven - Departement Werktuigkunde.
Khalaquzzaman, M., Kang, H. G., Kim, M. C., & Seong, P. H. (2011). Optimization of periodic testing frequency of a reactor protection system based on a risk-cost model and public risk perception. Nuclear Engineering and Design, 241(5), 1538–1547. doi:10.1016/j.nucengdes.2011.02.003
Laouti, N., Sheibat-Othman, N., & Othman, S. (2011). Support vector machines for fault detection in wind turbines. In Proceedings of IFAC World Congress (pp. 7067–7072).
Latorella, K. A., & Prabhu, P. V. (2000). A review of human error in aviation maintenance and inspection. International Journal of Industrial Ergonomics, 26(2), 133–161.
e o, . P., it gibbon, K. T., Puttini, . C., de Melo, G. P. B. (2008). Cost-benefit analysis methodology for PHM applied to legacy commercial aircraft. In IEEE Aerospace Conference (p. 12).
Lee, J. D., & Moray, N. (1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35(10), 1243–1270.
Lee, J. D., & Moray, N. (1994). Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies, 40(1), 153–184.
Lee, J. D., & See, K. A. (2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46(1), 50–80.
Lee, J., Ghaffari, M., & Elmeligy, S. (2011). Self-maintenance and engineering immune systems: Towards smarter machines and manufacturing systems. Annual Reviews in Control, 35(1), 111–122. doi:10.1016/j.arcontrol.2011.03.007
Lee, J., Kladwang, W., Lee, M., Cantu, D., Azizyan, M., Kim, H., … Das, R. (2014). RNA design rules from a massive open laboratory. In Proceedings of the National Academy of Sciences (pp. 2–7). doi:10.1073/pnas.1313039111
Lee, J., & Lapira, E. (2014). Recent Advances and Trends in Predictive Manufacturing in Industry 4.0 Environment. Uptime Magazine, 16–21.
Madhavan, P., & Wiegmann, D. A. (2007a). Effects of information source, pedigree, and reliability on operator interaction with decision support systems. Human Factors, 49(5), 5773–5785.
Madhavan, P., & Wiegmann, D. A. (2007b). Similarities and differences between human-human and human-automation trust: An integrative review. Theoretical Issues in Ergonomics Science, 8(4), 277–301.
Martignon, L., Krauss, S. (2003). Can l’homme eclaire be fast and frugal? Reconciling Bayesianism and bounded rationality. In S. Schneider & J. Shanteau (Eds.), Emerging perspectives on judgment and decision research (pp. 108–122). Cambridge: Cambridge University Press.
Mathur, A., Cavanaugh, K. F., Pattipati, K. R., Willett, P. K., & Galie, T. R. (2001). Reasoning and modeling systems in diagnosis and prognosis. In P. K. Willett & T. Kirubarajan (Eds.), Proceedings of SPIE—The International Society for Optical Engineering: Component and Systems Diagnostics, Prognosis, and Health Management (Vol. 4389, pp. 194–203). Orlando, Florida. doi:10.1117/12.434239
Merritt, S. M., & Ilgen, D. R. (2008). Not all trust is created equal: Dispositional and history based trust in human-automation interactions. Human Factors, 50(2), 194–210.
Mintzberg, H., & Simon, H. a. (1977). The New Science of Management Decision, Revised Edition. Administrative Science Quarterly, 22(2), 342. doi:10.2307/2391966
Montes de Oca, S., Puig, V., & Blesa, J. (2012). Robust fault detection based on adaptive threshold generation using interval LPV observers. International Journal of Adaptive Control and Signal Processing, 26(3), 258–283. doi:10.1002/acs.1263
Muir, . M., Moray, N. (1989). Operators’ trust in and use of automatic controllers. In Proceedings of the 22nd Annual Conference of the Human Factors Association of Canada (pp. 163–166). Ontario: Human Factors Association of Canada.
Pecht, M. G. (2008). Prognostics and Health Management of Electronics (p. 300). Hoboken, New Jersey: John Wiley & Sons, Inc.
Popov, A., Fink,, W., & Hess, A. (2013). PHM for Astronauts – A New Application. In S. Sankararaman & I. Roychoudhury (Eds.), Annual Conference of the Prognostics and Health Management Society 2013 (pp. 566–572). New Orleans LA: PHM Society. Retrieved from http://www.phmsociety.org/sites/phmsociety.org/files/phm_submission/2013/phmc_13_083.pdf
ProcessIT Europe. (2013). European Roadmap for Industrial Process Automation (p. 52).
Puig, V., Quevedo, J., Escobet, T., Nejjari, F., & de las Heras, S. (2008). Passive robust fault detection of dynamic processes using interval models. IEEE Transactions on Control Systems Technolog, 16, 1083–1089.
Rankin, W., Hibit, R., Allen, J., & Sargent, R. (2000). Development and evaluation of the Maintenance Error Decision Aid (MEDA) process. International Journal of Industrial Ergonomics, 26(2), 261–276. doi:10.1016/S0169-8141(99)00070-0
Rasmussen, N. C. (1975). Reactor safety study. An assessment of accident risks in U. S. commercial nuclear power plants. Executive Summary. WASH-1400 (NUREG-75/014) (p. 10). Rockville, MD, USA.
Sackett, D. L., Rosenberg, W. M. C., Gray, J. A. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: what it is and what it isn’t. BMJ, 312(7023), 71–72.
Salvendy, G. (Ed.). (2012). Handbook of Human Factors and Ergonomics (4th ed., p. 1752). Hoboken, NJ, USA: John Wiley & Sons, Inc. doi:10.1002/0470048204
Sandborn, P. (2005). A decision support model for determining the applicability of prognostic health management (PHM) approaches to electronic systems. In Annual Reliability and Maintainability Symposium, 2005. Proceedings. (pp. 422–427). Arlington, VA: IEEE. doi:10.1109/RAMS.2005.1408399
Sankararaman, S., & Goebel, K. (2012). Why is the Remaining Useful ife Prediction Uncertain ? In Annual Conference of the Prognostics and Health Management Society 2013 (pp. 1–13). New Orleans, LA: PHM Society.
Sarter, N. B., & Schroeder, B. (2001). Supporting Decision Making and Action Selection under Time Pressure and Uncertainty: The Case of In-Flight Icing. Human Factors, 43(4), 573–583.
Sarter, N. B., Woods, D. D., & Billings, C. E. (1997). Automation surprises. In G. Salvendy (Ed.), Handbook of Human Factors and Ergonomics (2nd ed., pp. 1926–1943). New York: John Wiley & Sons, Inc.
Saxena, A., Roychoudhury, I., & Celaya, J. R. (2010). Requirements Specifications for Prognostics: An Overview. In AIAA Infotech @ Aerospace (pp. 1–16). Atlanta, Georgia.
Sheridan, T. B. (1999). Human supervisory control. In A. P. Sage & W. B. Rouse (Eds.), Handbook of Systems Engineering and Management (pp. 645–690). New York: John Wiley & Sons, Inc.
Sheridan, T. B., & Parasuraman, R. (2006). Human-automation interaction. In R. S. Nickerson (Ed.), Reviews of human factors and ergonomics (1st ed., pp. 89–129). Santa Monica, CA: Human Factors and Ergonomics Society.
Shibl, R., Lawley, M., & Debuse, J. (2013). Factors influencing decision support system acceptance. Decision Support Systems, 54(2), 953–961. doi:10.1016/j.dss.2012.09.018
Souza, L. M. S. de, Spiess, P., Guinard, D., Moritz, K., & Karnouskos, S. (2008). SOCRADES : A Web Service Based Shop Floor Integration Infrastructure. In C. Floerkemeier, M. Langheinrich, E. Fleisch, F. Mattern, & S. E. Sarma (Eds.), Internet of Things 2008 Conference (pp. 50–67). Zurich, Switzerland.
Teti, R., Jemielniak, K., O’Donnell, G., Dornfeld, D. (2010). Advanced monitoring of machining operations. CIRP Annals - Manufacturing Technology, 59(2), 717–739. doi:10.1016/j.cirp.2010.05.010
The Nuclear Installations Inspectorate of the Health and Safety Executive. (2010). Safety Assessment Principles for Nuclear Facilities Technical Assessment Guide T/AST/058 - Human Factors Integration. Bootle, Merseyside: Office for Nuclear Regulation.
Uckun, S., Goebel, K., & Lucas, P. J. F. (2008). Standardizing research methods for prognostics. In 2008 International Conference on Prognostics and Health Management (pp. 1–10). Ieee. doi:10.1109/PHM.2008.4711437
Vachtsevanos, G., Lewis, F. L., Roemer, M., Hess, A., & Wu, B. (2006). Intelligent Fault Diagnosis and Prognosis for Engineering Systems (1st ed., p. 456). Hoboken, New Jersey: John Wiley & Sons, Inc. doi:10.1002/9780470117842.app1
Venkatasubramanian, V., Rengaswamy, R., & Kavuri, S. N. (2003). A review of process fault detection and diagnosis Part II : Qualitative models and search strategies. Computers & Chemical Engineering, 27(3), 313–326.
Venkatasubramanian, V., Rengaswamy, R., Yin, K., & Kavuri, S. N. (2003). A review of process fault detection and diagnosis Part I : Quantitative model-based methods. Computers & Chemical Engineering, 27(3), 293–311.
Walker, M., & Kapadia, R. (2009). Integrated Design of On-line Health and Prognostics Management. In Annual Conference of the Prognostics and Health Management Society (pp. 1–15). San Diego, CA.
Wickens, C. D., & Dixon, S. R. (2007). The benefits of imperfect diagnostic automation: a synthesis of the literature. Theoretical Issues in Ergonomics Science, 8(3), 201–212. doi:10.1080/14639220500370105
Wiegmann, D. A., Rich, A. M., & Zhang, H. (2001). Automated diagnostic aids: The effects of aid reliability on users’ trust and reliance. Theoretical Issues in Ergonomics Science, 2(4), 352–367.
Wiener, E. L., & Curry, R. E. (1980). Flight-deck automation: Promises and problems. Ergonomics, 23(10), 995–1011.
Yam, R. C. M., Tse, P. W., Li, L., & Tu, P. (2001). Intelligent Predictive Decision Support System for Condition-Based Maintenance. The International Journal of Advanced Manufacturing Technology, 17(5), 383–391. doi:10.1007/s001700170173
Yu, B. Y., Syed Zubair, M. H. S., & Yang, M. C. (2013). A Framework for System Design Optimization Based On Maintenance Scheduling With Prognostics and Health Management. In Proceedings of ASME 2013 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (p. 12). Portland, Oregan.
Zhao, F., Tian, Z., & Zeng, Y. (2013). Uncertainty Quantification in Gear Remaining Useful Life Prediction Through an Integrated Prognostics Method. IEEE Transactions on Reliability, 62(1), 14.
Zhao, W., Zio, E., & Baraldi, P. (2011). Confidence in signal reconstruction by the Evolving Clustering Method. In 2011 Prognostics and System Health Managment Confernece (pp. 1–7). IEEE. doi:10.1109/PHM.2011.5939535
Zio, E. (2012). Prognostics and Health Management of Industrial Equipment. In S. Kadry (Ed.), Diagnostics and Prognostics of Engineering Systems: Methods and Techniques (pp. 333–356). IGI Global. doi:10.4018/978-1-4666-2095-7
The Prognostic and Health Management Society advocates open-access to scientific data and uses a Creative Commons license for publishing and distributing any papers. A Creative Commons license does not relinquish the author’s copyright; rather it allows them to share some of their rights with any member of the public under certain conditions whilst enjoying full legal protection. By submitting an article to the International Conference of the Prognostics and Health Management Society, the authors agree to be bound by the associated terms and conditions including the following:
As the author, you retain the copyright to your Work. By submitting your Work, you are granting anybody the right to copy, distribute and transmit your Work and to adapt your Work with proper attribution under the terms of the Creative Commons Attribution 3.0 United States license. You assign rights to the Prognostics and Health Management Society to publish and disseminate your Work through electronic and print media if it is accepted for publication. A license note citing the Creative Commons Attribution 3.0 United States License as shown below needs to be placed in the footnote on the first page of the article.
First Author et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 United States License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.