96 Part II: Multi-state deterioration modelling Sunny Cloudy Rainy 0.7 0.2 0.1 0.3 0.4 0.3 0.2 0.3 0.5 (a) Discrete-time Markov chain. (b) State probability over time. Figure II.1: Markov chain modelling the weather condition (example). in Sunny decreases while it increases for the other states. As noted later in this section, Markov chains model random processes and are therefore useful for sampling; for example, from Figure II.1(a), the following sequence could be obtained: Sunny ↑Rainy ↑Rainy ↑Cloudy ↑· · ·. The definitions below were adopted from Stewart, 2009; Brémaud, 2020; Colombo, Abreu, and Martins, 2021. Definition 6 (Markov chain). Let a stochastic process be defined by a family of random variables Xt ↓ T, where t represents time and X denotes the value of the random variable at time t; T ⇔R is the parameter space. If T is discrete, the process is discrete-time; if T is continuous, the process is continuous-time. The values assumed by Xt are termed states, which may be either continuous or discrete; in this dissertation, we use the latter. A Markov chain is defined as a tuple M=↔S, Pij, p (0)↗ where: - S is a countable set called the state space which contains all possible states of the chain. - Pij : S↘S ↑[0,1] is a family of transition probabilities P(t, ϱ; i, j) such that for any states i, j ↓ S, time t ↓ T, and future time ϱ ↓ T where ϱ ⇒ t, P(t, ϱ; i, j) = P(Xε = j | Xt = i) gives the probability of transitioning from state i at time t to state j at time ϱ. Additionally, Pij ⇒0 for all i, j ↓S, and )j→S Pij =1 for all i ↓S. - p(0) : S ↑[0,1] is the initial probability distribution over S. This distribution satisfies p (0) k ⇒0 for all k ↓ S and )k →S p (0) k =1. If Pij depends on the time the process was initiated, it is termed aninhomogeneoustime stochastic process; on the other hand, if Pij do not depend on the time the process was initiated, it is termed a homogeneous-time stochastic process. If Xt is invariant under any shift of the time origin, it is termed a stationary process; if it varies with the time of initiation, it is termed non-stationary. Xt has the Markov property if it satisfies the following definition.
RkJQdWJsaXNoZXIy MjY0ODMw