II.4.1 Markov chains 97 Definition 7 (Markov property). The Markov property states that the conditional probability of transitioning to any future state depends only on the present state and not on the sequence of events that preceded it. Formally, for any n-length sequence of times ↔t0 ⇓t1 ⇓· · · ⇓tn↗ in T, and states ↔x0, x1, . . . , xn↗ in S, the property is given by: P(Xt =x| Xt1 =x1, Xt2 =x2, . . . , Xtn =xn) =P(Xt =x| Xtn =xn), (4.1) this equation holds for all states and time points in the index set T. In this dissertation, we use inhomogeneous, homogeneous, continuous, and discretetime Markov chains. Before providing their formal definitions and for completeness, below in Definition 8 we describe probability density function, survival function, and hazard rate (more in Appendix C.1): Definition 8 (Stochastic Process Functions). For a stochastic process characterised by a set of functions parametrised by ς—scale, shape, or other distribution-specific parameters—the key functions are: - Probability Density Function (PDF), f(t; ς): describes the probability density of a continuous random variable at a specific value t, where f(t; ς) ⇒0 for all t and / f(t; ς)dt =1 across the domain of t. - Survival Function (SF), S(t; ς): represents the probability that the lifetime of a system or component exceeds a specific time t, formally defined as S(t; ς) = 1→/t 0 f(x; ς)dx. Note that S(·) is also referred to as the reliability function in other fields. - Hazard Rate (HR), ϖ(t; ς): rate at which failures occur given survival until time t. It is defined by the ratio ϖ(t; ς)= f(t;ϑ) S(t;ϑ), assuming S(t; ς) >0. Inhomogeneous Time Markov Chain In an Inhomogeneous Time Markov Chain (IHTMC) the transitions between states are time-dependent and it is formally defined as follows. Definition 9 (Inhomogeneous-Time Markov Chain). An Inhomogeneous Time Markov Chain is defined by the tuple M=↔S, p(0), Qij(t, ς)↗, see S and p (0) in Definition 6. Here: - Qij(t, ς) : S↘S ↑R is a time-dependent transition rate matrix. This matrix function, parametrised by time t and parameters ς, is structured as follows: – The non-diagonal entries qij(t, ς), for i, j ↓ S and i ⇑ = j, represent the transition rate from state i to state j at time t. – The diagonal entries qii(t, ς) are chosen to ensure that each row sum of Qij is zero, signifying that the rate out of any state is equal to the sum of the rates into other states.
RkJQdWJsaXNoZXIy MjY0ODMw