168 Chapter 7. Maintenance Strategies for Sewer Pipes with Multi-State Deterioration and Deep Reinforcement Learning Agent-E’s, potentially explaining why Agent-G is better equipped to navigate and understand the deterioration pattern. This, in turn, enables it to devise a more e!ective maintenance policy by leveraging a more accurate deterioration model. 7.7.3 Policy analysis over episode In Section 7.7.2, we present an overview of policy performances. This section delves into the details per episode to provide further understanding on these policies. Figures 7.4, 7.5, and 7.6 detail the performance of the Agent-E, Agent-G, CBM, and SchM policies for pipes with ages 0, 25 and 50, respectively. The RM heuristic is excluded from this analysis due to its straightforward approach: allowing the pipe to fail before replacing it. k=1 k=2 k=3 k=4 k=5 k=F Total Policy Costs:. -€2.0K [Maintenance: €-2.0K, Replace: €0.0K, Failure: €0.0K] Total Reward: -0.00 0.00 0.25 0.50 0.75 1.00 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Time horizon Action at =0 at =1 at =2 (a) Agent-G k=1 k=2 k=3 k=4 k=5 k=F 0.00 0.25 0.50 0.75 1.00 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Time horizon Action at =0 at =1 at =2 Total Policy Costs:. -€24.6K [Maintenance: €0.0K, Replace: -€24.6K, Failure: €0.0K] Total Reward: -0.20 (b) Agent-E k=1 k=2 k=3 k=4 k=5 k=F 0.00 0.25 0.50 0.75 1.00 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Time horizon Action at =0 at =1 at =2 Total Policy Costs.: €0.0K [Maintenance: €0.0K, Replace: €0.0K, Failure: €0.0K] Total Reward: 0.00 (c) Condition-based Maintenance (CBM) k=1 k=2 k=3 k=4 k=5 k=F 0.00 0.25 0.50 0.75 1.00 Total Policy Costs:. -€6.0K [Maintenance: -€6.0K, Replace: €0.0K, Failure: €0.0K] Total Reward: -0.00 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Time horizon Action at =0 at =1 at =2 (d) Scheduled Maintenance (SchM) Figure 7.4: Policies behaviour over an episode for a new pipe, showing the health vector over the pipe age and actions per policy: (a) Agent-G, (b) Agent-E, (c) Condition-based Maintenance (CBM), and (d) Scheduled Maintenance (SchM).
RkJQdWJsaXNoZXIy MjY0ODMw