9 9.2. Recommendations 191 studies. 8. System-level dependencies: Future research should incorporate systemlevel dependencies—such as stochastic and structural (de Jonge and Scarf, 2020)—as treating the sewer network as a collection of ‘disconnected’ elements is not realistic. The influence of these dependencies on degradation assessment and modelling is largely unexplored, and their inclusion could significantly enhance the accuracy of the models. 9. Hybrid methods: Data-driven approaches alone may not su"ce, especially considering the persistence of data quality issues. Hybrid methods that combine data-driven models with other types of models (e.g., physics-based or expert knowledge) could enhance performance and help overcome data limitations. These hybrid approaches could be especially useful when data is scarce or of poor quality. 9.2.3 Strategic Maintenance Planning for Sewer Mains using Reinforcement Learning In Chapter 7, we explored Deep Reinforcement Learning (DRL) for component-level strategic maintenance in sewer mains. Key challenges and future research directions include: 1. Account for context: Context refers to static variables in the state space that influence the reward and transition functions. For example, di!erent materials used in sewer mains may result in varying degradation profiles. The maintenance strategy should adapt to such contextual factors. For large systems with variable context, frameworks like Contextual Markov Decision Process (Hallak, Di Castro, and Mannor, 2015), which extend standard Markov Decision Process to incorporate contextual variability, could be beneficial. 2. Partial observability: The assumption of a fully observable state space is unrealistic, as not all component states are visible at all times. Shifting to partially observable state spaces, where inspections are required to reveal component states, expands the optimisation problem. This allows the planning of inspections to gather data and improve decision-making. Frameworks such as Partially Observable Markov Decision Processes (Kıvanç, Özgür-Ünlüakın, and Bilgiç, 2022) o!er potential solutions for addressing this issue. 3. Hyper-parameter tuning: A significant challenge in DRL is tuning the RL algorithm’s hyper-parameters, which can greatly influence agent behaviour. This includes both the parameters of the MDP and the hyper-parameters specific to DRL algorithms. Selecting the ‘right’ hyper-parameters is crucial, as they impact the agents’ ability to adopt the desired maintenance strategies. Research into more e!ective tuning methods is needed to ensure agents exhibit optimal behaviour.
RkJQdWJsaXNoZXIy MjY0ODMw