172 Chapter 7. Maintenance Strategies for Sewer Pipes with Multi-State Deterioration and Deep Reinforcement Learning - Advancing toward partially observable state spaces with the introduction of inspection actions, considering context, and leveraging deep learning capabilities. - Using the knowledge acquired by the agents to develop explainable and robust heuristics. - Although this chapter focused on a single cohort of pipes, studies in Jimenez-Roa, Heskes, Tinga, et al., 2022; Jimenez-Roa, Tinga, Heskes, et al., 2024 show di!erent cohorts exhibit varied dynamics, highlighting the importance of understanding how RL agents adapt. - Comparing RL-based approaches with other policy optimisation algorithms to better understand the capacity of RL methods to achieve global-optima maintenance strategies. - Investigating various reward functions (e.g., dense) and RL algorithms to determine the most e!ective for devising maintenance policies. - Extent to system-level analysis and evaluate scalability. - Moving toward multi-infrastructure asset management to promote coordinated management for optimising costs and minimising disruption from interventions. 7.9 References Akiba, T., S. Sano, T. Yanase, T. Ohta, and M. Koyama (2019). “Optuna: A Nextgeneration Hyperparameter Optimization Framework”. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. KDD ’19. Anchorage, AK, USA: Association for Computing Machinery, pp. 2623–2631. isbn: 9781450362016. doi: 10.1145/3292500.3330701. Ansel, J., E. Yang, H. He, N. Gimelshein, A. Jain, M. Voznesensky, B. Bao, P. Bell, D. Berard, E. Burovski, G. Chauhan, A. Chourdia, W. Constable, A. Desmaison, Z. DeVito, E. Ellison, W. Feng, J. Gong, M. Gschwind, B. Hirsh, S. Huang, K. Kalambarkar, L. Kirsch, M. Lazos, M. Lezcano, Y. Liang, J. Liang, Y. Lu, C. K. Luk, B. Maher, Y. Pan, C. Puhrsch, M. Reso, M. Saroufim, M. Y. Siraichi, H. Suk, S. Zhang, M. Suo, P. Tillet, X. Zhao, E. Wang, K. Zhou, R. Zou, X. Wang, A. Mathews, W. Wen, G. Chanan, P. Wu, and S. Chintala (2024). “PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation”. In: Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. ASPLOS ’24. La Jolla, CA, USA: Association for Computing Machinery, pp. 929–947. doi: 10.1145/3620665.3640366. de Jonge, B. and P. A. Scarf (2020). “A review on maintenance optimization”. In: European Journal of Operational Research 285.3, pp. 805–824. issn: 0377-2217. doi: 10.1016/j.ejor.2019.09.047. Jimenez-Roa, L. A., T. Heskes, T. Tinga, H. J. A. Molegraaf, and M. Stoelinga (2022). “Deterioration modeling of sewer pipes via discrete-time Markov chains: A largescale case study in the Netherlands”. In: Proceedings of the 32nd European Safety and Reliability Conference, ESREL 2022 - Understanding and Managing Risk and Reliability for a Sustainable Future, pp. 1299–1306. doi: 10.3850/978-981-18-5183-4_R22-13482-cd. Jimenez-Roa, L. A., T. Tinga, T. Heskes, and M. Stoelinga (2024). “Comparing Homogeneous and Inhomogeneous Time Markov Chains for Modelling Degradation in Sewer Pipe Networks”. In: Proceedings of the European Safety and Reliability Conference (ESREL 2024). doi: 10.48550/arXiv.2407.12557.
RkJQdWJsaXNoZXIy MjY0ODMw