Adaptive Multi-scale Prognostics and Health Management for Smart Manufacturing Systems
##plugins.themes.bootstrap3.article.main##
##plugins.themes.bootstrap3.article.sidebar##
Published
Nov 13, 2020
Benjamin Y. Choo
Stephen C. Adams
Brian A. Weiss
Jeremy A. Marvel
Peter A. Beling
Abstract
The Adaptive Multi-scale Prognostics and Health Management (AM-PHM) is a methodology designed to enable PHM in smart manufacturing systems. In application, PHM information is not yet fully utilized in higher-level decisionmaking in manufacturing systems. AM-PHM leverages and integrates lower-level PHM information such as from a machine or component with hierarchical relationships across the component, machine, work cell and assembly line levels in a manufacturing system. The AM-PHM methodology enables the creation of actionable prognostic and diagnostic intelligence up and down the manufacturing process hierarchy. Decisions are then made with the knowledge of the current and projected health state of the system at decision points along the nodes of the hierarchical structure. To overcome the issue of exponential explosion of complexity associated with describing a large manufacturing system, the AM-PHM methodology takes a hierarchical Markov Decision Process (MDP) approach into describing the system and solving for an optimized policy. A description of the AM-PHM methodology is followed by a simulated industry-inspired example to demonstrate the effectiveness of AM-PHM.
##plugins.themes.bootstrap3.article.details##
Keywords
Markov Decision Process, smart manufacturing, hierarchical MDP
References
AlDurgam, M. M., & Duffuaa, S. O. (2013). Optimal joint maintenance and operation policies to maximise overall systems effectiveness. International Journal of Production Research, 51(5), 1319–1330.
Amari, S. V., McLaughlin, L., & Pham, H. (2006). Costeffective condition-based maintenance using markov decision processes. In Reliability and maintainability symposium, 2006. rams’06. annual (pp. 464–469).
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. (n.d.). , 112.
Bouvard, K., Artus, S., Berenguer, C., & Cocquempot, V. (2011). Condition-based dynamic maintenance operations planning & grouping. application to commercial heavy vehicles. Reliability Engineering & System Safety, 96(6), 601–610.
Byon, E., & Ding, Y. (2010). Season-dependent conditionbased maintenance for a wind turbine using a partially observed markov decision process. Power Systems, IEEE Transactions on, 25(4), 1823–1834.
Byon, E., Ntaimo, L., & Ding, Y. (2010). Optimal maintenance strategies for wind turbine systems under stochastic weather conditions. Reliability, IEEE Transactions on, 59(2), 393–404.
Chan, G., & Asgarpoor, S. (2006). Optimum maintenance policy with markov processes. Electric Power Systems Research, 76(6), 452–456.
Chen, D., & Trivedi, K. S. (2005). Optimization for condition-based maintenance with semi-markov decision process. Reliability Engineering & System Safety, 90(1), 25–29.
Choo, B. Y., Beling, P. A., LaViers, A. E., Marvel, J. A., & Weiss, B. A. (2015). Adaptive multi-scale phm for robotic assembly processes. In Proceedings of the annual conference of the prognostics and health management society.
Davis, J., Edgar, T., Porter, J., Bernaden, J., & Sarli, M. (2012, December). Smart manufacturing, manufacturing intelligence and demand-dynamic performance. Computers & Chemical Engineering, 47, 145–156.
de Jonge, B., Klingenberg, W., Teunter, R., & Tinga, T. (2016). Reducing costs by clustering maintenance activities for multiple critical units. Reliability engineering & system safety, 145, 93–103.
Dietterich, T. G. (2000, November). Hierarchical reinforcement learning with the maxq value function decomposition. J. Artif. Int. Res., 13(1), 227–303.
Grall, A., Bérenguer, C., & Dieulle, L. (2002). A conditionbased maintenance policy for stochastically deteriorating systems. Reliability Engineering & System Safety, 76(2), 167–180.
Hameed, Z., Hong, Y., Cho, Y., Ahn, S., & Song, C. (2009). Condition monitoring and fault detection of wind turbines and related algorithms: A review. Renewable and Sustainable energy reviews, 13(1), 1–39.
Holdren, J. P. (2011). Report to the President on Ensuring American Leadership in Advanced Manufacturing. President’s Council of Advisors on Science and Technology.
Hopp, W. J., & Spearman, M. L. (2011). Factory physics (3. ed., reissued ed.). Long Grove, Ill: Waveland Press.
Huynh, K. T., Barros, A., & Berenguer, C. (2015). Multilevel decision-making for the predictive maintenance of k-out-of-n: F deteriorating systems. Reliability, IEEE Transactions on, 64(1), 94–117.
Jardine, A. K., Lin, D., & Banjevic, D. (2006). A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical systems and signal processing, 20(7), 1483–1510.
Lam, C. T., & Yeh, R. (1994). Optimal maintenance-policies for deteriorating systems under various maintenance strategies. Reliability, IEEE Transactions on, 43(3), 423–430.
Lee, J., Wu, F., Zhao, W., Ghaffari, M., Liao, L., & Siegel, D. (2014). Prognostics and health management design for rotary machinery systems – Reviews, methodology and applications. Mechanical Systems and Signal Processing, 42(1), 314–334.
Lu, B., Li, Y., Wu, X., & Yang, Z. (2009). A review of recent advances in wind turbine condition monitoring and fault diagnosis. In Power electronics and machines in wind applications, 2009. pemwa 2009. ieee (pp. 1–7).
Maillart, L. M. (2006). Maintenance policies for systems with condition monitoring and obvious failures. IIE Transactions, 38(6), 463–475.
Marvel, J. A. (2014). Collaborative Robotics: A Gateway into Factory Automation. ThomasNet News.
Montgomery, N., Lindquist, T., Garnero, M.-A., Chevalier, R., & Jardine, A. (2006). Reliability functions and optimal decisions using condition data for EDF primary pumps. In Probabilistic methods applied to power systems, 2006. pmaps 2006. international conference on (pp. 1–6).
Nguyen, K.-A., Do, P., & Grall, A. (2015). Multi-level predictive maintenance for multi-component systems. Reliability Engineering & System Safety, 144, 83–94.
Parr, R., & Russell, S. (1998). Reinforcement learning with hierarchies of machines. In Proceedings of the 1997 conference on advances in neural information processing systems 10 (pp. 1043–1049).
Peng, Y., Dong, M., & Zuo, M. J. (2010). Current status of machine prognostics in condition-based maintenance: a review. The International Journal of Advanced Manufacturing Technology, 50(1-4), 297–313.
Powell, W. B. (2011). Approximate dynamic programming: Solving the curse of dimensionality (Second ed.). John Wiley & Sons.
Robelin, C.-A., & Madanat, S. M. (2007). History-dependent bridge deck maintenance and replacement optimization with markov decision processes. Journal of Infrastructure Systems, 13(3), 195–201.
Shafiee, M., & Finkelstein, M. (2015). An optimal age-based group maintenance policy for multi-unit degrading systems. Reliability Engineering & System Safety, 134, 230–238.
Si, X.-S., Wang, W., Hu, C.-H., & Zhou, D.-H. (2011). Remaining useful life estimation–a review on the statistical data driven approaches. European Journal of Operational Research, 213(1), 1–14.
Sikorska, J., Hodkiewicz, M., & Ma, L. (2011). Prognostic modelling options for remaining useful life estimation by industry. Mechanical Systems and Signal Processing, 25(5), 1803–1836.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge, Mass: MIT Press.
Tomasevicz, C. L., & Asgarpoor, S. (2009). Optimum maintenance policy using semi-markov decision processes. Electric Power Systems Research, 79(9), 1286–1291.
Van Horenbeek, A.,&Pintelon, L. (2013). A dynamic predictive maintenance policy for complex multi-component systems. Reliability Engineering & System Safety, 120, 39–50.
Vogl, G. W., Weiss, B. A., & Donmez, M. A. (2014). Standards for Prognostics and Health Management (PHM) Techniques within Manufacturing Operations. In Annual Conference of the Prognostics and Health Management Society.
Wu, B., Tian, Z., & Chen, M. (2013). Condition-based maintenance optimization using neural network-based health condition prediction. Quality and Reliability Engineering International, 29(8), 1151–1163.
Yam, R., Tse, P., Li, L., & Tu, P. (2001). Intelligent predictive decision support system for condition-based maintenance. The International Journal of Advanced Manufacturing Technology, 17(5), 383–391.
Zhang, J., & Lee, J. (2011). A review on prognostics and health monitoring of Li-ion battery. Journal of Power Sources, 196(15), 6007–6014.
Amari, S. V., McLaughlin, L., & Pham, H. (2006). Costeffective condition-based maintenance using markov decision processes. In Reliability and maintainability symposium, 2006. rams’06. annual (pp. 464–469).
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. (n.d.). , 112.
Bouvard, K., Artus, S., Berenguer, C., & Cocquempot, V. (2011). Condition-based dynamic maintenance operations planning & grouping. application to commercial heavy vehicles. Reliability Engineering & System Safety, 96(6), 601–610.
Byon, E., & Ding, Y. (2010). Season-dependent conditionbased maintenance for a wind turbine using a partially observed markov decision process. Power Systems, IEEE Transactions on, 25(4), 1823–1834.
Byon, E., Ntaimo, L., & Ding, Y. (2010). Optimal maintenance strategies for wind turbine systems under stochastic weather conditions. Reliability, IEEE Transactions on, 59(2), 393–404.
Chan, G., & Asgarpoor, S. (2006). Optimum maintenance policy with markov processes. Electric Power Systems Research, 76(6), 452–456.
Chen, D., & Trivedi, K. S. (2005). Optimization for condition-based maintenance with semi-markov decision process. Reliability Engineering & System Safety, 90(1), 25–29.
Choo, B. Y., Beling, P. A., LaViers, A. E., Marvel, J. A., & Weiss, B. A. (2015). Adaptive multi-scale phm for robotic assembly processes. In Proceedings of the annual conference of the prognostics and health management society.
Davis, J., Edgar, T., Porter, J., Bernaden, J., & Sarli, M. (2012, December). Smart manufacturing, manufacturing intelligence and demand-dynamic performance. Computers & Chemical Engineering, 47, 145–156.
de Jonge, B., Klingenberg, W., Teunter, R., & Tinga, T. (2016). Reducing costs by clustering maintenance activities for multiple critical units. Reliability engineering & system safety, 145, 93–103.
Dietterich, T. G. (2000, November). Hierarchical reinforcement learning with the maxq value function decomposition. J. Artif. Int. Res., 13(1), 227–303.
Grall, A., Bérenguer, C., & Dieulle, L. (2002). A conditionbased maintenance policy for stochastically deteriorating systems. Reliability Engineering & System Safety, 76(2), 167–180.
Hameed, Z., Hong, Y., Cho, Y., Ahn, S., & Song, C. (2009). Condition monitoring and fault detection of wind turbines and related algorithms: A review. Renewable and Sustainable energy reviews, 13(1), 1–39.
Holdren, J. P. (2011). Report to the President on Ensuring American Leadership in Advanced Manufacturing. President’s Council of Advisors on Science and Technology.
Hopp, W. J., & Spearman, M. L. (2011). Factory physics (3. ed., reissued ed.). Long Grove, Ill: Waveland Press.
Huynh, K. T., Barros, A., & Berenguer, C. (2015). Multilevel decision-making for the predictive maintenance of k-out-of-n: F deteriorating systems. Reliability, IEEE Transactions on, 64(1), 94–117.
Jardine, A. K., Lin, D., & Banjevic, D. (2006). A review on machinery diagnostics and prognostics implementing condition-based maintenance. Mechanical systems and signal processing, 20(7), 1483–1510.
Lam, C. T., & Yeh, R. (1994). Optimal maintenance-policies for deteriorating systems under various maintenance strategies. Reliability, IEEE Transactions on, 43(3), 423–430.
Lee, J., Wu, F., Zhao, W., Ghaffari, M., Liao, L., & Siegel, D. (2014). Prognostics and health management design for rotary machinery systems – Reviews, methodology and applications. Mechanical Systems and Signal Processing, 42(1), 314–334.
Lu, B., Li, Y., Wu, X., & Yang, Z. (2009). A review of recent advances in wind turbine condition monitoring and fault diagnosis. In Power electronics and machines in wind applications, 2009. pemwa 2009. ieee (pp. 1–7).
Maillart, L. M. (2006). Maintenance policies for systems with condition monitoring and obvious failures. IIE Transactions, 38(6), 463–475.
Marvel, J. A. (2014). Collaborative Robotics: A Gateway into Factory Automation. ThomasNet News.
Montgomery, N., Lindquist, T., Garnero, M.-A., Chevalier, R., & Jardine, A. (2006). Reliability functions and optimal decisions using condition data for EDF primary pumps. In Probabilistic methods applied to power systems, 2006. pmaps 2006. international conference on (pp. 1–6).
Nguyen, K.-A., Do, P., & Grall, A. (2015). Multi-level predictive maintenance for multi-component systems. Reliability Engineering & System Safety, 144, 83–94.
Parr, R., & Russell, S. (1998). Reinforcement learning with hierarchies of machines. In Proceedings of the 1997 conference on advances in neural information processing systems 10 (pp. 1043–1049).
Peng, Y., Dong, M., & Zuo, M. J. (2010). Current status of machine prognostics in condition-based maintenance: a review. The International Journal of Advanced Manufacturing Technology, 50(1-4), 297–313.
Powell, W. B. (2011). Approximate dynamic programming: Solving the curse of dimensionality (Second ed.). John Wiley & Sons.
Robelin, C.-A., & Madanat, S. M. (2007). History-dependent bridge deck maintenance and replacement optimization with markov decision processes. Journal of Infrastructure Systems, 13(3), 195–201.
Shafiee, M., & Finkelstein, M. (2015). An optimal age-based group maintenance policy for multi-unit degrading systems. Reliability Engineering & System Safety, 134, 230–238.
Si, X.-S., Wang, W., Hu, C.-H., & Zhou, D.-H. (2011). Remaining useful life estimation–a review on the statistical data driven approaches. European Journal of Operational Research, 213(1), 1–14.
Sikorska, J., Hodkiewicz, M., & Ma, L. (2011). Prognostic modelling options for remaining useful life estimation by industry. Mechanical Systems and Signal Processing, 25(5), 1803–1836.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge, Mass: MIT Press.
Tomasevicz, C. L., & Asgarpoor, S. (2009). Optimum maintenance policy using semi-markov decision processes. Electric Power Systems Research, 79(9), 1286–1291.
Van Horenbeek, A.,&Pintelon, L. (2013). A dynamic predictive maintenance policy for complex multi-component systems. Reliability Engineering & System Safety, 120, 39–50.
Vogl, G. W., Weiss, B. A., & Donmez, M. A. (2014). Standards for Prognostics and Health Management (PHM) Techniques within Manufacturing Operations. In Annual Conference of the Prognostics and Health Management Society.
Wu, B., Tian, Z., & Chen, M. (2013). Condition-based maintenance optimization using neural network-based health condition prediction. Quality and Reliability Engineering International, 29(8), 1151–1163.
Yam, R., Tse, P., Li, L., & Tu, P. (2001). Intelligent predictive decision support system for condition-based maintenance. The International Journal of Advanced Manufacturing Technology, 17(5), 383–391.
Zhang, J., & Lee, J. (2011). A review on prognostics and health monitoring of Li-ion battery. Journal of Power Sources, 196(15), 6007–6014.
Section
Technical Papers