160 Chapter 7. Maintenance Strategies for Sewer Pipes with Multi-State Deterioration and Deep Reinforcement Learning Quantifying Deterioration: Each segment’s deterioration level is initially assessed and categorized into severity levels according to the MSDM (M). As the deterioration progresses, the state of each segment transitions according to the probabilities described by the matrix Pij, where i is the current severity level and j is the subsequent severity level. This transition process is governed by the forward Kolmogorov equation (referenced as Eq. 4.2 on page 98). Notice that by doing this, we assume there is no statistical dependency between segments, which is a strong assumption that needs further research. However, for simplicity, we maintain this assumption in our deterioration model. The severity levels for each segment in the sewer main are captured by the vector d¯ (Eq. 7.3), with the length of d¯ equal to ω. Each component di within this vector denotes the current severity level of its corresponding segment. d¯ =(d1, d2, . . . , dω) where di̸ M, (7.3) to quantify this distribution in the health vector h, we first count the number of segments at each severity level k using the following expression: 1k = ω i=1 1 {d¯i=k} for each k ↓S (7.4) where 1 is the indicator function that is 1 if the condition is true and 0 otherwise. The health vector h is then determined by normalising these counts to reflect the proportion of segments at each severity level: hk = 1k ω , (7.5) where 1k is the number of segments at severity level k. Thus, hk becomes part of the state space indicating the level of deterioration present in the pipe. Stochastic prediction of severity levels To enable the agent to access information provided by the MSDM, we incorporate the prediction of severity levels into the state space. This is accomplished by solving Eq. 4.3, yielding a distribution pk(t). Finally, our state space is defined as a tuple with 13 elements: S =↔Pipe Age, h1, h2, h3, h4, h5, hF, p1, p2, p3, p4, p5, pF↗
RkJQdWJsaXNoZXIy MjY0ODMw