6.2.1 Multi-state deterioration modelling for sewer networks using parametrised Markov chains . . . . . . . . . . . . . . . . . . 126 6.2.2 Modelcalibration........................ 127 6.2.3 Non-parametric modelling . . . . . . . . . . . . . . . . . . . 129 6.2.4 Goodness-of-fit metrics . . . . . . . . . . . . . . . . . . . . . 130 6.3 Experimental setup and evaluation . . . . . . . . . . . . . . . . . . 130 6.3.1 Casestudy ........................... 130 6.3.2 Experimentalsetup....................... 130 6.3.3 Results ............................. 131 6.4 Findings................................. 132 6.4.1 Comparison between cohorts . . . . . . . . . . . . . . . . . 132 6.4.2 Transition probabilities over time . . . . . . . . . . . . . . . 132 6.4.3 Overfitting ........................... 132 6.4.4 Comparing inhomogeneous Markov chains . . . . . . . . . . 132 6.5 Conclusions and future research . . . . . . . . . . . . . . . . . . . . 134 6.6 References................................ 135 Part III: Maintenance optimisation of multi-state components 138 III.1 Introduction.............................. 139 III.2 Nomenclature ............................. 140 III.3 Relatedwork ............................. 141 III.4 Preliminaries ............................. 145 III.4.1 Markov Decision Process . . . . . . . . . . . . . . . . . . . 145 III.4.2 Deep Reinforcement Learning . . . . . . . . . . . . . . . . 145 III.4.3 Contextual Markov Decision Process . . . . . . . . . . . . 147 III.4.4 Proximal Policy Optimisation . . . . . . . . . . . . . . . . 148 7 Maintenance Strategies for Sewer Pipes with Multi-State Deterioration and Deep Reinforcement Learning 153 Abstract................................. 153 7.1 Introduction............................... 154 7.2 Technicalbackground ......................... 155 7.3 Methodology .............................. 155 7.4 Multi-state deterioration models . . . . . . . . . . . . . . . . . . . 156 7.4.1 Casestudy ........................... 156 7.4.2 Parametrisation......................... 156 7.4.3 Solving the Multi-State Deterioration Model . . . . . . . . 157 7.4.4 Parametric Multi-State Deterioration Models . . . . . . . . 157 7.5 Definition of Markov Decision Process for Maintenance Policy Optimisation of a sewer main considering deterioration over the pipe length .................................. 158 7.5.1 State space S .......................... 159
RkJQdWJsaXNoZXIy MjY0ODMw