170 Chapter 7. Maintenance Strategies for Sewer Pipes with Multi-State Deterioration and Deep Reinforcement Learning shortening the maintenance intervals; (b) Agent-E postpones any action until the pipe fails, at which point it replaces the pipe with a new one, akin to RM; (c) CBM also initiates maintenance around the pipe’s 50-year mark. However, deterioration escalates from age 60, leading to failure at 66. The inability to manage this increased severity results in significant penalty costs, diminishing the e!ectiveness of this policy; (d) Similarly, SchM manages severity levels e!ectively until the pipe reaches approximately 70 years of age, at which point deterioration accelerates, resulting in failure at 73. k=1 k=2 k=3 k=4 k=5 k=F 0.00 0.25 0.50 0.75 1.00 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Time horizon Action at =0 at =1 at =2 Total Policy Costs.: -€33.8K [Maintenance: -€9.2K, Replace: -€24.6K, Failure: €0.0K] Total Reward: -0.20 (a) Agent-G k=1 k=2 k=3 k=4 k=5 k=F 0.00 0.25 0.50 0.75 1.00 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Time horizon Action at =0 at =1 at =2 Total Policy Costs:. -€24.6K [Maintenance: €0.0K, Replace: -€24.6K, Failure: €0.0K] Total Reward: -0.20 (b) Agent-E k=1 k=2 k=3 k=4 k=5 k=F 0.00 0.25 0.50 0.75 1.00 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Total Policy Costs:. -€32.1K [Maintenance: -€7.5K, Replace: -€24.6K, Failure: €0.0K] Total Reward: -0.20 Time horizon Action at =0 at =1 at =2 (c) Condition-based Maintenance (CBM) k=1 k=2 k=3 k=4 k=5 k=F 0.00 0.25 0.50 0.75 1.00 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Time horizon Action at =0 at =1 at =2 Total Policy Costs.: -€149.7K [Maintenance: -€25.1K, Replace: -€24.6K, Failure: -€100.0K] Total Reward: -1.10 (d) Scheduled Maintenance (SchM) Figure 7.6: Policies behaviour over an episode for a pipe aged 50, showing the health vector over the pipe age and actions per policy: (a) Agent-G, (b) Agent-E, (c) Conditionbased Maintenance (CBM), and (d) Scheduled Maintenance (SchM). Figure 7.6 shows that for a pipe aged 50: (a) Agent-G opts to replace the pipe at age 50, followed by maintenance in the subsequent time step. This decision is likely influenced by parts of the pipe being at severity levels k ↓3,4. Such a
RkJQdWJsaXNoZXIy MjY0ODMw