Review of Markov Models for Maintenance Optimization in the Context of Offshore Wind



Rafael Dawid David McMillan Matthew Revie


The offshore environment poses a number of challenges to wind farm operators. Harsher climatic conditions typically result in lower reliability while challenges in accessibility make maintenance difficult. One of the ways to improve availability is to optimize the Operation and Maintenance (O&M) actions such as scheduled, corrective and proactive maintenance. Many authors have attempted to model or optimize O&M through the use of Markov models. Two examples of Markov models, Hidden Markov Models (HMMs) and Partially Observable Markov Decision Processes (POMDPs) are investigated in this paper. In general, Markov models are a powerful statistical tool, which has been successfully applied for component diagnostics, prognostics and maintenance optimization across a range of industries. This paper discusses the suitability of these models to the offshore wind industry. Existing models which have been created for the wind industry are critically reviewed and discussed. As there is little evidence of widespread application of these models, this paper aims to highlight the key factors required for successful application of Markov models to practical problems. From this, the paper identifies the necessary theoretical and practical gaps that must be resolved in order to gain broad acceptance of Markov models to support O&M decision making in the offshore wind industry.

How to Cite

Dawid, R., McMillan, D., & Revie, M. (2015). Review of Markov Models for Maintenance Optimization in the Context of Offshore Wind. Annual Conference of the PHM Society, 7(1).
Abstract 5 | PDF Downloads 4



wind energy, maintenance optimization, POMDP, Markov models, Deterioration modelling, Hidden Markov Model

AlDurgam, M. M., & Duffuaa, S. O. (2012). Optimal joint maintenance and operation policies to maximise overall systems effectiveness. International Journal of Production Research, (July 2015), 1–12. doi:10.1080/00207543.2012.659351
Amari, S. V., & McLaughlin, L. (2006). Cost-effective condition-based maintenance using Markov decision processes. RAMS ’06. Annual Reliability and Maintainability Symposium, 2006, 00(C), 464–469. doi:10.1109/RAMS.2006.1677417
Berenguer, C., Chu, C., & Grall, A. (1997). Inspection and maintenance planning : an application of semi-Markov decision processes. Journal of Intelligent Manufacturing. Besnard, F., & Bertling, L. (2010). An Approach for Condition-Based Maintenance Optimization Applied to Wind Turbine Blades. IEEE Transactions on Sustainable Energy, 1(2), 77–83. doi:10.1109/TSTE.2010.2049452
Black, M., Brint, a T., & Brailsford, J. R. (2005). A semiMarkov approach for modelling asset deterioration. Journal of the Operational Research Society, 56(11), 1241–1249. doi:10.1057/palgrave.jors.2601967
Byon, E., & Ding, Y. (2010). Season-Dependent ConditionBased Maintenance for a Wind Turbine Using a Partially Observed Markov Decision Process. IEEE Transactions on Power Systems, 25(4), 1823–1834. doi:10.1109/TPWRS.2010.2043269
Byon, E., Ntaimo, L., & Ding, Y. D. Y. (2010). Optimal Maintenance Strategies for Wind Turbine Systems Under Stochastic Weather Conditions. IEEE Transactions on Reliability, 59(2), 393–404. doi:10.1109/TR.2010.2046804
Byon, E. (2012). Wind turbine operations and maintenance: A tractable approximation of dynamic decision-making. IIE Transactions, 120918112955009. doi:10.1080/0740817X.2012.726819
Cartella, F., Lemeire, J., Dimiccoli, L., & Sahli, H. (2015). Hidden Semi-Markov Models for Predictive Maintenance. Mathematical Problems in Engineering, 2015. doi:
Chan, G. K., & Asgarpoor, S. (2006). Optimum maintenance policy with Markov processes. Electric Power Systems Research, 76(6-7), 452–456. doi:10.1016/j.epsr.2005.09.010 Chen, D., & Trivedi, K. S. (2005). Optimization for condition-based maintenance with semi-Markov decision process. Reliability Engineering and System Safety, 90, 25–29. doi:10.1016/j.ress.2004.11.001
Chen, M., Fan, H., Hu, C., & Zhou, D. (2014). Maintaining Partially Observed Systems With Imperfect Observation and Resource Constraint. IEEE Transactions on Reliability, 63(4), 881–890.
Chinnam, R. B., & Baruah, P. (2003). Autonomous diagnostics and prognostics through competitive learning driven HMM-based clustering. Proceedings of the International Joint Conference on Neural Networks, 2003., 4, 2466–2471. doi:10.1109/IJCNN.2003.1223951
Cho, D. I., & Parlar, M. (1991). A survey of maintenance models for multi- unit systems. European Journal of Operational Researc, 51, 1–23.
Cibulka, J., Ebbesen, M. K., Hovland, G., Robbersmyr, K. G., & Hansen, M. R. (2012). A review on approaches for condition based maintenance in applications with induction machines located offshore. Modeling, Identification and Control, 33(2), 69–86.
Corotis, R. B., Ellis, H. J., & Jiang, M. (2005). Modeling of risk-based inspection, maintenance and life-cycle cost with partially observable Markov decision processes. Structure and Infrastructure Engineering, 1(1), 75–84. doi:10.1080/15732470412331289305
David, I., Friedman, L., & Sinuany-Stern, Z. (1999). A simple suboptimal algorithm for system maintenance under partial observability. Annals of Operations Research, 91, 25–40. Dekker, R. (1996). Applications of maintenance optimization models: a review and analysis. Reliability Engineering & System Safety, 51, 229–240. doi:10.1016/0951-8320(95)00076-3 Dong, M. (2008). A novel approach to equipment health management based on auto-regressive hidden semiMarkov model (AR-HSMM). Science in China, Series F: Information Sciences, 51(9), 1291–1304. doi:10.1007/s11432-008-0111-4
Dong, M., & He, D. (2007). A segmental hidden semiMarkov model (HSMM)-based diagnostics and prognostics framework and methodology. Mechanical Systems and Signal Processing, 21(5), 2248–2266. doi:10.1016/j.ymssp.2006.10.001
Dong, M., He, D., Banerjee, P., & Keller, J. (2006). Equipment health diagnosis and prognosis using hidden semi-Markov models. International Journal of Advanced Manufacturing Technology, 30(7-8), 738749. doi:10.1007/s00170-005-0111-0
Douard, F., Domecq, C., & Lair, W. (2012). A probabilistic approach to introduce risk measurement indicators to an offshore wind project evaluation - Improvement to an existing tool ECUME. Energy Procedia, 255–262. doi:10.1016/j.egypro.2012.06.107
Dragomir, O. E., Gouriveau, R., Dragomir, F., Minca, E., & Zerhouni, N. (2009). Review of Prognostic Problem in Condition-Based Maintenance. European Control Conference. El-Thalji, I. (2012). On the operation and maintenance practices of wind power asset: A status review and observations. Journal of Quality in Maintenance Engineering, 18, 232–266. Fan, H., Xu, Z., & Chen, S. (2013). Optimally Maintaining a Multi-State System with Limited Imperfect Preventive Repairs. International Journal of Systems Science, 00(0), 1–12. doi:10.1080/00207721.2013.828799
Feng, Y., Tavner, P. J., & Long, H. (2004). Early experiences with UK round 1 offshore wind farms. Journal of Business Ethics, 44(November), 0–103. doi:10.1063/1.2756072 Fischer, K., Besnard, F., & Bertling, L. (2012). ReliabilityCentered Maintenance for Wind Turbines Based on Statistical Analysis and Practical Experience. Ieee Transactions on Energy Conversion, 27(1), 184–195. doi:10.1109/TEC.2011.2176129
Frangopol, D. M., Kallen, M.-J., & Noortwijk, J. M. Van. (2004). Probabilistic models for life-cycle performance of deteriorating structures: review and future directions. Progress in Structural Engineering and Materials, 6(4), 197–212. doi:10.1002/pse.180
Ghasemi, A., Yacout, S., & Ouali, M. (2007). Optimal replacement policy and observation interval for CBM with imperfect information. World Congress on Engineering and Computer Science.
GL Garrad Hassan. (2013). A Guide to UK Offshore Wind Operations and Maintenance. Scottish Enterprise and The Crown Estate.
Hameed, Z., Hong, Y. S., Cho, Y. M., Ahn, S. H., & Song, C. K. (2009). Condition monitoring and fault detection of wind turbines and related algorithms: A review. Renewable and Sustainable Energy Reviews, 13, 1–39. doi:10.1016/j.rser.2007.05.008
Heng, A., Zhang, S., Tan, A. C. C., & Mathew, J. (2009). Rotating machinery prognostics: State of the art, challenges and opportunities. Mechanical Systems and Signal Processing, 23(3), 724–739. doi:10.1016/j.ymssp.2008.06.009
Hofmann, M. (2011). A Review of Decision Support Models for Offshore Wind Farms with an Emphasis on Operation and Maintenance Strategies. Wind Engineering, 35, 1–16. doi:10.1260/0309-524X.35.1.1
Ivy, J. S., & Pollock, S. M. (2005). Marginally monotonic maintenance policies for a multi-state deteriorating machine with probabilistic monitoring, and silent failures. IEEE Transactions on Reliability, 54(3), 489497. doi:10.1109/TR.2005.853443 Jiang,
M., Corotis, R. B., & Ellis, J. H. (2000). Optimal Life-Cycle Costing with Partial Observability. Journal of Infrastructure Systems, 56–66.
Johnson, A., Strachan, S., & Ault, G. (2012). A Framework for Asset Replacement and Investment Planning in Power Distribution Networks. Asset Management Conference 2012, IET & IAM, 1–5.
Kahrobaee, S., & Asgarpoor, S. (2013). A hybrid analyticalsimulation approach for maintenance optimization of deteriorating equipment: Case study of wind turbines. Electric Power Systems Research, 104, 80–86. doi:10.1016/j.epsr.2013.06.012
Kharoufeh, J. P., Solo, C. J., & Ulukus, M. Y. (2010). SemiMarkov models for degradation-based reliability. IIE Transactions, 42(8), 599–612. doi:10.1080/07408170903394371
Kleiner, Y. (2001). Scheduling Inspection and Renewal of Large Infrastructure Assets. Journal of Infrastructure Systems, 7(4), 136–143. doi:10.1061/(ASCE)10760342(2001)7:4(136) Kothamasu, R., Huang, S. H., & Verduin, W. H. (2009). System health monitoring and prognostics - A review of current paradigms and practices. Handbook of Maintenance Management and Engineering, 337–362. doi:10.1007/978-1-84882-472-0_14
Kwan, C., Zhang, X., Xu, R., & Haynes, L. (2003). A Novel Approach to Fault Diagnostics and Prognostics. Proceedings of the 2003 lEEE International Conference on Robotics & Automation, 604–609.
Lee, J., Ni, J., Sarangapani, J., Mathew, J. (2011). A Review of Machinery Diagnostics and Prognostics Implemented on a Centrifugal Pump.
WCEAM, 594. Lee, S., Li, L., & Ni, J. (2013). Markov-Based Maintenance Planning Considering Repair Time and Periodic Inspection. Journal of Manufacturing Science and Engineering, 135(3), 031013. doi:10.1115/1.4024152
Madanat, S. (1993). Optimal infrastructure management decisions under uncertainty. Transportation Research Part C, 1(I), 77–88. doi:DOI: 10.1016/0968090X(93)90021-7
Maillart, L. M. (2006). Maintenance policies for systems with condition monitoring and obvious failures. IIE Transactions, 38(6), 463–475. doi:10.1080/074081791009059
Makis, V., & Jiang, X. (2003). Optimal Replacement Under Partial Observations. Mathematics of Operations Research, 28(February 2015), 382–394. doi:10.1287/moor.28.2.382.14484 Maksoud, E. Y. a, & Moustafa, M. S. (2009). A semiMarkov decision algorithm for the optimal maintenance of a multistage deteriorating two-unit standby system. Operational Research, 9, 167–182. doi:10.1007/s12351008-0022-6
McMillan, D., & Ault, G. W. (2007). Condition Monitoring Benefit for Onshore Wind Turbines: Sensitivity to Operational Parameters. Renewable Power Generation, IET, 1(1), 10–16. doi:10.1049/iet-rpg
McMillan, D., & Ault, G. W. (2009). Quantification of Condition Monitoring Benefit for Offshore Wind Turbines. Wind Engineering, 31(0), 267–285. doi:10.1260/030952407783123060 Memarzadeh, M., Pozzi, M., & Kolter, J. Z. (2013). Optimal Planning and Learning in Uncertain Environments for the Management of Wind Farms. Journal of Computing in Civil Engineering, 27(5), 511–521. doi:10.1061/(ASCE)CP
Monahan, G. E. (1982). State of the Art--A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms. Management Science, 28(1), 1–16. doi:10.1287/mnsc.28.1.1
Nicolai, R. P., & Dekker, R. (2008). Optimal Maintenance of Multi-component Systems : A Review. Complex System Maintenance Handbook, (1991), 263–286. doi:10.1007/978-1-84800-011-7_11
Nielsen, J. S., & Sørensen, J. D. (2012). Maintenance Optimization for Offshore Wind Turbines using POMDP. Reliability and Optimization of Structural Systems, American University of Armenia Press, Yrevan, Armenia, page 175–182.

Nielsen, J., & Sørensen, J. (2014). Methods for Risk-Based Planning of O&M of Wind Turbines. Energies, 7(10), 6645–6664. doi:10.3390/en7106645
Özdirik, B., Skiba, M., Würtz, F., Kaltschmitt, M., & Williams, P. (2013). O & M Modelling for Large Scale Offshore Wind Farms by Use of Markov Processes. Pahlke, T. (2007). Software & Decision Support Systems for Offshore Wind Energy Exploitation in the North Sea Region.

Papakonstantinou, K. G., & Shinozuka, M. (2014a). Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: Theory. Reliability Engineering and System Safety, 130, 214–224. doi:10.1016/j.ress.2014.04.005

Papakonstantinou, K. G., & Shinozuka, M. (2014b).Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation. Reliability Engineeringand System Safety, 130, 214–224. doi:10.1016/j.ress.2014.04.006
Papakonstantinou, K. G., & Shinozuka, M. (2014c).Optimum inspection and maintenance policies for corroded structures using partially observable Markov decision Processes and stochastic, physically based models. Probabilistic Engineering Mechanics, 37, 93108. doi:10.1016/j.probengmech.2014.06.002

Peng, Y., Dong, M., & Zuo, M. J. (2010). Current status of machine prognostics in condition-based maintenance: A review. International Journal of Advanced Manufacturing Technology, 50(1-4), 297–313. doi:10.1007/s00170-009-2482-0

Pierskalla, W. P., & Voekler, J. A. (1976). A Survey of Maintenance Models: The Control and Surveillance of Deteriorating Systems.

Qian, S., Jiao, W., Hu, H., & Yan, G. (2007). Transformer power fault diagnosis system design based on the HMM method. Proceedings of the IEEE International Conference on Automation and Logistics, ICAL 2007, (2), 1077–1082. doi:10.1109/ICAL.2007.4338728

Reinertsen, R. (1996). Residual life of technical systems; diagnosis, prediction and life extension. Reliability Engineering and System Safety, 54(1), 23–34. doi:10.1016/S0951-8320(96)00092-0
Rosenfield, D. (1976). Markovian Deterioration with Uncertain Information. Operations Research, 24(1), 141–155. doi:10.1287/opre.24.1.141
Scarf, P. a. (1997). On the application of mathematical models in maintenance. European Journal of Operational Research, 99(3), 493–506. doi:10.1016/S0377-2217(96)00316-5 Shafiee, M. (2015). Maintenance logistics organization for offshore wind energy: Current progress and future perspectives. Renewable Energy, 77, 182–193. doi:10.1016/j.renene.2014.11.045
Sherif, Y. S., & Smith, M. L. (1981). Optimal Maintenance Models for Systems Subject to Failure - A Review.

Si, X. S., Wang, W., Hu, C. H., & Zhou, D. H. (2011).Remaining useful life estimation - A review on the statistical data driven approaches. European Journal of Operational Research, 213(1), 1–14. doi:10.1016/j.ejor.2010.11.018

Srinivasan, R., & Parlikad, a K. N. (2014). Semi Markov decision process with partial information for optimum maintenance decisions. IEEE Transactions on Reliability, 63(4), 891–898.

Tai, A. H., Ching, W. K., & Chan, L. Y. (2009). Detection of machine failure: Hidden Markov Model approach. Computers and Industrial Engineering, 57(2), 608–619. doi:10.1016/j.cie.2008.09.028
Tung, T. Van, & Yang, B.-S. (2009). Machine Fault Diagnosis and Prognosis: The State of The Art. International Journal of Fluid Machinery and Systems, 2, 61–71. doi:10.5293/IJFMS.2009.2.1.061U.S.

DoE, (2008). Energy Efficiency and Renewable Energy; 20% Wind Energy by 2030. Increasing Wind Energy's contribution to US Electric Supply (Report no. DOE/GO-102008–2567) Retrieved from:

Valdez-Flores, C., & Feldman, R. M. (1989). A Survey of Preventive Maintenance Models for Stochiastically Deteriorating Single-Unit Systems.

Van Horenbeek, A., Van Ostaeyen, J., Duflou, J. R., & Pintelon, L. (2013). Quantifying the added value of an imperfectly performing condition monitoring system Application to a wind turbine gearbox. Reliability Engineering and System Safety, 111, 45–57. doi:10.1016/j.ress.2012.10.010

Vasili, M., Hong, T., & Ismail, N. (2011). Maintenance optimization models: a review and analysis. International Conference on Industrial Engineering and Operations Management, 1131–1138. Retrieved from
Wang, H. (2002). A survey of maintenance policies of deteriorating systems. European Journal of Operational Research, 139(3), 469–489. doi:10.1016/S03772217(01)00197-7

Welte, T. M., & Wang, K. (2013). Models for Lifetime Estimation - An Overview with Focus on Applications to Wind Turbines. Iwama, 337–350. doi:10.1007/s40436-014-0064-3

White, C. C. (1976). Procedures for the Solution of a FiniteHorizon, Partially Observed, Semi-Markov Optimization Problem. Operations Research, 24(2), 348–358. doi:10.1287/opre.24.2.348
Wijnmalen, D. J. D., & Hontelez, J. a. M. (1992). Review of a Markov decision algorithm for optimal inspections and revisions in a maintenance system with partial information. European Journal of Operational Research, 62, 96–104. doi:10.1016/0377-2217(92)90180-H

Wilson, G., & McMillan, D. (2014). Assessing Wind Farm Reliability Using Weather Dependent Failure Rates. Journal of Physics: Conference Series, 524, 012181. doi:10.1088/1742-6596/524/1/012181 Y.
Zhou, L. Ma, J. Matthew, Y. Sun, R. Wolff. (2010).Maintenance Decision-Making Using a ContinuousState Partially Observable Semi-Markov Decision Process.

Yang, F., Kwan, C. M., & Chang, C. S. (2008).Multiobjective evolutionary optimization of substation maintenance using decision-varying Markov model. IEEE Transactions on Power Systems, 23(3), 13281335. doi:10.1109/TPWRS.2008.922637
Zhang, X., Xu, R., Kwan, C., Liang, S. Y., Xie, Q., & Haynes, L. (2005). An integrated approach to bearing fault diagnostics and prognostics. American Control Conference, 2005. Proceedings of the 2005, 27502755. doi:10.1109/ACC.2005.1470385

Zhong, C., & Jin, H. (2014). A novel optimal preventive maintenance policy for a cold standby system basedon semi-Markov theory. European Journal of Operational Research, 232(2), 405–411. doi:10.1016/j.ejor.2013.07.020
Zhou, Z. J., Hu, C. H., Xu, D. L., Chen, M. Y., & Zhou, D. H. (2010). A model for real-time failure prognosis based on hidden Markov model and belief rule base. European Journal of Operational Research, 207(1), 269–283. doi:10.1016/j.ejor.2010.03.032
Zio, E. (2009). Reliability engineering: Old problems and new challenges. Reliability Engineering & System Safety, 94(2), 125–141. doi:10.1016/j.ress.2008.06.002
Technical Papers