8 8.3. Maintenance Optimisation: Maintenance optimisation of multi-state components 183 mains, which is the focus of Part III. Obtaining optimal maintenance policies is challenging, especially in large state spaces, where computing the optimal policy can be ine"cient or infeasible within time constraints. This motivated the exploration of alternatives like Deep Reinforcement Learning (DRL). While DRL does not guarantee globally optimal solutions, we aimed to assess its potential for Maintenance Policy Optimisation (MPO), focusing on how degradation model assumptions a!ect agent performance. Recap on contributions in Part III In Part III, we used Multi-State Deterioration Model (MSDM) and Deep Reinforcement Learning to devise component-level strategic maintenance planning with applications in sewer mains. Our contributions are two-fold: 1. In Chapter 7, we propose a DRL framework for devising maintenance policies at the pipe level, considering MSDM. We detail model calibration and have made our models and dataset publicly available in the repository: zenodo.org/records/11258904. 2. We evaluate the influence of homogeneous and inhomogeneous MSDM on devising strategic maintenance, comparing agent behaviours against wellknown maintenance policy heuristics. Our findings suggest that DRL o!ers a flexible framework with untapped potential for maintenance strategies, and it is crucial to integrate degradation model assumptions, as they significantly influence policy behaviour. When does it make sense to use DRL for maintenance optimisation? DRL extends Reinforcement Learning (RL), providing a flexible framework to tackle sequential decision-making problems, producing virtual agents with enforced behaviours through ‘trial and error’. This approach is particularly suitable for maintenance optimisation when the state space is large. For systems like sewer networks, large state spaces arise easily due to the number of discrete and continuous variables involved (e.g., covariates, sensor data, etc.). Thus, e!ective techniques to ‘solve’ these optimisation problems are crucial. Approaches that seek globally optimal solutions through exhaustive search can be computationally expensive. This is where alternative approaches, such as DRL, can explore larger state spaces, though without guaranteeing global optimality. Both approaches are valuable, and further research with applications on these type of systems remains important. Notably, a key challenge in DRL is the time-consuming hyper-parameter tuning needed to achieve agents with ‘good behaviour’.
RkJQdWJsaXNoZXIy MjY0ODMw