166 Chapter 7. Maintenance Strategies for Sewer Pipes with Multi-State Deterioration and Deep Reinforcement Learning Table 7.4 presents the mean policy cost for Agent-E, Agent-G, CBM, SchM, and RM, highlighting the best and second-best policies in blue and red, with corresponding means and standard deviations from the simulations. From these results, we observe that Agent-G’s policy generally outperforms others for pipe ages of 0 and 25 years, securing a second-best position for pipes aged 50 years. It is noted that the cost of all policies increases with pipe age, which aligns with expectations as older pipes require more interventions. After reviewing the mean policy costs, our focus shifts to the specific actions involved in each policy. Table 7.5 provides a summary of the actions executed by each policy across simulations for di!erent pipe ages. For new pipes, the SchM policy leads in maintenance activities (at =1), with Agent-G following. In terms of replacements (at =2), Agent-E is the foremost in implementing this action, with CBM in second place. Both Agent-G and SchM exhibit lower replacement frequencies, explaining the mean policy costs since maintenance actions incur lower expenses compared to the penalties and replacement costs resulting from pipe failures. Table 7.5: Percentage (%) of actions per policy obtained with Agent-E, Agent-G, CBM, SchM, and RM, evaluated over 100 episodes in the test environment, for di!erent pipe ages. Pipe age Action Agent-E Agent-G CBM SchM RM 0 at =0 99.5% 97.51% 99.54% 94.76% 99.61% at =1 0.0% 2.21% 0.05% 4.95% 0.00% at =2 0.5% 0.28% 0.41% 0.29% 0.39% 25 at =0 98.81% 94.96 % 98.14% 94.56% 98.92% at =1 0.00% 4.50% 0.62% 4.94% 0.00% at =2 1.19% 0.53% 1.24% 0.50% 1.08% 50 at =0 98.4% 94.52% 98.05% 93.99% 98.68% at =1 0.0% 4.43% 0.67% 4.88% 0.00% at =2 1.6% 1.05% 1.28% 1.13% 1.32% For pipes aged 25 years, Agent-G executes more maintenance actions (at =1), similar to SchM. Agent-E opts for no maintenance, aligning more with RM’s strategy. Although CBM carries out some maintenance actions, replacement actions predominate, indicating a greater tendency to permit pipe failures, which explains the observed di!erences in mean policy costs. For pipes aged 50 years, CBM o!ers the most cost-e!ective policy, with Agent-G’s following. CBM conducts fewer maintenance actions and more replacements than Agent-G, accounting for the cost disparity. The policies of Agent-E, RM, and SchM have similar costs. Despite SchM conducting more maintenance, its high number of replacements suggests the maintenance interval requires adjustment. These results
RkJQdWJsaXNoZXIy MjY0ODMw