Stochastic models of just-in-time systems and windows of vulnerability in terms of the processes of birth and death

Full Text

Introduction In the paper, we propose a method for constructing models based on an anal- ysis of the processes of birth and death with linear growth in semimartingale terms. As the main examples of stochastic models, simple just-in-time systems and windows of vulnerability are considered. Just-in-time systems (abbreviated as JIT are analyzed in the theory of productive systems, and windows of vulnera- bility (abbreviated as WoV are widely discussed in risk theory. The main results obtained in the work are presented in terms of the average values of the time during which the processes reach zero values. The work also considers (and uses when studying models) estimates of the local times of the studied processes. We note that here, simple Markov processes with a linear growth of intensities (perhaps, depending on time) are analyzed. Nevertheless, the obtained and used estimates are of a certain theoretical interest. Thus, for example, the average value of the stopping time, at which the process reaches zero, depends on functions such as the harmonic number and the remainder term for the logarithmic function in the Taylor theorem. However, as one of the main results, it is proposed to consider the presented method of mathematical modeling of JIT and WoV systems. The semimartingale description approach used here should be considered as the first step of such a modeling, since, being a trajectory method, it allows diffusion (including non- Markov processes) generalizations when constructing stochastic models of WoV and JIT. At present, when constructing probabilistic models of WoV, in the theory of risk, researches have appeared that analyzes the average values of the time during which the trajectories are in a certain area and therefore the system is vulnerable (see, for example [1-3] and referen- ces in them). Studies of average values of the local time for such models are interesting. And if the WoV descriptions in the risk theory are predominantly stochastic, then the models based on JIT are mostly deterministic. Although it is obvious that a probabilistic approach to the study of such processes is in demand. We also note that the stochastic description of JIT systems proposed here is in a certain sense a logical (and analytic) continuation of [4], in which a semimartingale description is given for problems of optimal control of such systems. 526 In this article, statements for the average values of the local time and the stop- ping times when the birth and death processes reach a given value are obtained. This allows us to consistently present estimates for the models of the JIT system and for WoV, the result for which is given in the form of a limit theorem. The principle of JIT is well known and used in many areas. However, its emer- gence and development is associated primarily with the analysis of productive systems (see, for example, [5-9], as well as [4] and referen- ces therein). Here, as in cite [4], we touch on the time reversal method. A number of papers are devoted to the study of this method for stochastic systems represented in the semimartingale description (see, for example, [10-12] and referen- ces therein). Also close to this method are studies of stochastic bridges (see, for example, [13-15] and referen- ces therein). Descriptions in terms of the processes of birth and death are devoted primarily to models in biology (see, for example, [16]). However, the semimartin- gale approach allows for significant generalizations. So, it allows one to describe and investigate models of random walk processes for a wide class of objects (see, e.g., [17-20] and [4]). All the main results in the paper are formulated as theorems and lemmas. The proofs of the results use semimartingale methods. 1. Preliminaries and necessary theoretical results 1.1. Notations, definitions and assumptions Let B = (, F , F = (F t ) t>0 , P) denote a stochastic basis, that is, a probability space (, F , P) with a nondecreasing right-continuous family of -algebras F = (F t ) t>0 , complete with respect to P (see e.g., the conditions of [21]). Notation 1. For any special locally square-integrable semimartingale U = = ( U t ) t>0 and m U = (m U (U t ) t>0 on the basis B, we denote by U t ) t>0 the compen- t + m U sator of U, and the local martingale, respectively: U t = U t . For definiteness, U we also assume that U 0 = U 0 and m 0 = 0. The predictable process of locally , and the martingale m U are locally square-integrable. We bounded variation ( U ) U denote by m = m U t t>0 the predictable quadratic characteristic of m U . Notation 2. For any process of birth and death U on B with strictly positive initial value U 0 > 1 we denote by U = U (), , the Markov time at which U reaches zero: U = inf {t > 0 : U t = 0} (where inf{} = +). Notation 3. We denote by L U (K) the function of K N = {1, 2, . . .}, which is the average value of the stopping time U for the process U with the initial value U 0 = K: L U (K) = E U . Definition 1. Let l t U (n) be the local time of any process of birth and death U on B: for n N 0 and t R + = [0, +) : l t U (n) = t I{U s = n} ds, 0 where I{·} is an indicator function (i.e., I{true} = 1, I{false} = 0). Consider on the basis B a locally square-integrable birth and death process X = (X t ) t>0 with trajectories in the Skorokhod space, X t () N 0 = {0, 1, 2, . . .}, 527 with X 0 () = K N and X t () = X t () X t () {1, 0, 1} P-a.s. We can represent X as a difference X = X 0 + A B (see, for example, [18] and [19]), with X t = X 0 + A t B t , (1) where A = (A t ) t>0 and B = (B t ) t>0 are the counting processes of the number of positive and negative jumps of X, respectively: I{X s = 1} for all t > 0 P-a.s. I{X s = 1}, B t = A t = 0<s6t 0<s6t with the initial values A 0 = B 0 = 0 and X 0 = K > 1. Assumption 1. Suppose that X is a process of birth and death with linear growth, that is, the compensators of the submartingales A and B on B are t t · X s ds and B t = · X s ds for all t > 0. (2) A t = 0 0 Then the quadratic characteristics of the locally square-integrable martingales m A = (m A t ) t>0 and m B = (m B t ) t>0 are t , m A t = A t m B t = B and m A , m B t = 0 for all t > 0. (3) Assumption 2. We suppose that > 0, > 0 and > . It is clear that under the representation (2) and (3) the process X with the definition (1) is a square-integrable birth and death process with linear growth. For such a process, it easily follows from Assumption 2 that lim E X t = lim E X t 2 = 0. t> t> Therefore, from Chebyshev’s inequality we obtain lim E I{X t > i} = lim P{X t > i} = 0 for all i N. t> Thus, t> (4) P{ X < } = 1 lim P{X t > 0} = 1, t> and X is the stopping time on B. Notation 4. For any process of birth and death U on B we denote by U (n) the average value of the local time on R + for n N : U U (n) = E l (n), where U l (n) = lim l t U (n) t> 528 and l t U (n) is set in Definition 1. From Definition 1 it follows that X (n) = (5) P{X s = n} ds. 0 Since for X under consideration, X () < P-a.s., then we have X I{X s > 1} ds = lim = t> 0 where l t X (n) = n=1 (6) X l (n), n=1 X l (n) = lim l t X (n). t> From (5), Notation 4 and (6) it follows that L X (K) = E X = (7) X (n). n=1 We define the following auxiliary functions. Definition 2. Let H(n) be the harmonic number (i.e. the partial sum of the harmonic series) for n N: H(n) = n 1 i=1 i . Definition 3. Let R n (x) be the remainder term in Taylor theorem of the func- tion log(1 x) for n-th order Taylor polynomial P n (x) of x (0, 1), n N: R n (x) = log(1 x) P n (x) = log(1 x) + x i = . i i n x i i=1 i=n+1 Definition 4. We define the following auxiliary constants: = , = / (where 0 < 6 , 0 6 < 1). 1.2. Theoretical results necessary for the study of models Theorem 1. For the average value of the stopping time X , L X (K) = { ( )} ( 1 ) ( ) K · H(K) + log 1 · R K if > 0 (8) and L X (K) = H(K) if = 0. (9) 529 It should be noted that the expression (8) in the statement of the theorem is rather unusual. As for (9), it can also be regarded as the limiting case of (8) for > 0. Corollary 1. If K = 1, then for the average value of the stopping time X , L X (K) = and L X (K) = ( 1 ) · log 1 1 if if > 0 (10) (11) = 0. The proof of the theorem is based on the equality (7) and on the following lemma, which is also of independent interest. Lemma 1. For the average value of the local time on R + for n N, n 1 ·n ·n X (n) = (nK) n ·n ·n if if n 6 K, (12) n > K. Corollary 2. If = 0, then for the average value of the local time on R + for n N, X { (n) = 1/(·n) if 0 if n 6 K, n > K. (13) 2. The problem of JIT The term just-in-time arose in connection with the need for a description of the so-called production systems (see, for example, [6, 9] and referen- ces in [4]). Suppose that a fixed strictly positive finite time moment T > 0 is given. Suppose also that the process of birth and death Z = (Z t ) t[0,T ] (or an integer- valued random walk for the general case) has the initial value Z 0 = K N. Definition 5. The process of birth and death Z is just-in-time T (abbreviated as JIT T), if Z T = 0 P-a.s. and P{Z t > 0} > 0 for any t [0, T ) (that is, P{ Z 6 T } = 1 and P{ Z > t} > 0 for any t [0, T )). The process of birth and death Z is just-in-time (abbreviated as JIT ) if there exists a strictly positive finite number T > 0 such that the process Z is JIT T. As a simple example, consider the linear pure death JIT T process V = (V t ) t[0,T ] with death rates that are properly time-dependent. Suppose that in a certain production system it is necessary to perform a set of homogeneous opera- tions with successively decreasing numbers K 0 = {K, K 1, . . . , 1, 0} exactly for a period of time T , where K N and T (0, +). Usually, operation number 0 is considered terminal and unproductive (for example, warehousing or packaging). For systems of this type, it is of interest problem to estimate the time of the beginning of the last operation (with the number 0), since the control possibilities of the system after this moment vanish. 530 Let the nonincreasing process V = (V t ) t[0,T ] with values in K 0 “indicates” the number of the operation that is executed at each moment t [0, T ]. We denote by (n) the execution time of each operation with number n K 0 . Then (n) = l T V (n), where l T V is the local time defined in Definition 1, for the process V . In these notations, the problem mentioned above is formulated in the same way as in the first section: find E V , where the moment V is defined in Notation 2 and is equal to (K) + . . . + (1) = T (0). Note also that the process with reversed time V = (V u ) u[0,T ] with val- ues V u = V (T u) is a point counting process provided V T = K. If the times {(0), (1), . . . , (K)} were independent and identically exponentially distributed random variables, then V would be a Poisson process. However, in this case the condition V T = K would not hold. The trajectories of the Poisson process for which this condition is fulfilled form a well-known Poisson bridge, see, for exam- ple, [12, 13] and the literature cited therein. The compensator of the process V has the form u K V r · I{r < T } dr. V u = T r 0 Then the process V can be considered as a Poisson process with reversed time with an initial value V 0 = K and with the condition V (T ) = 0. Its compensator is equal to t V s V t = K · I{s < T } ds. 0 T s Note that the condition V (T ) = 0 can also be achieved with other depen- dencies of the death rate on time. As an example, consider the process, which is a very simple generalization of the process with the specified compensator V = ( V t ) t[0,T ] . Suppose that the linear pure death process Z can be represented as the dif- ference Z = Z 0 B with Z t = Z 0 B t and t [0, T ], where B = (B t ) t[0,T ] is the counting process of numbers of negative jumps of Z : B t = I{Z s = 1} 0<s6t with the compensator t · t = B 0 K N s · I{s < T } ds, T s > 0. Consequently, the supermartingale Z has the representation Z t = K t Z s · 0 · I{s < T } ds + m Z t , T s (14) For the average value of the stopping time L Z (K) = E Z of the linear pure death process Z, with the intensity depending on time in accordance with the representation (14), the following statement holds. 531 Theorem 2. Let > 0. Then for the average value of the stopping time Z , K (i 1)! T L (K) = · ) . i ( 1 j=1 j + Z (15) i=1 Remark 1. Note that (15) can be represented in the form, which is to some extent close to (9) of Theorem 1: E Z = K 1 1 · · {n}, n n=1 where {n} = T n ( 1+ i=1 1 ) . ·i Corollary 3. If = 1, then for the average value of the local time on R + for K N, L Z (K) = T · K . K +1 Remark 2. This result coincides with a value that we can easily obtain for a description in reverse time. Thus, for the stopping time (1) of the first jump of the process in reverse time Z = (Z u ) u[0,T ] = (Z (T u) ) u[0,T ] (i.e., (0) = inf{u : u > 0, z u = 1}), we have the distribution function F (0) (t) = 1 ( T T t ) K . Therefore, for the average value of (0), we have T t dF (0) (t) = E (0) = 0 T , K +1 and L Z (K) = E Z = T E (0) = T · K . K +1 To prove Theorem 2, we define on a stochastic basis B the sequence of stopping times d(n) = inf{t [0, T ] : Z(t) = n} and the sequence of mean values (n) = E d(n), n K 0 , and note that for n K = {K, K 1, . . . , 1}, we have d(n 1) d(n) = l T Z (n) and (n 1) (n) = Z (n), (16) where Z (n) = E l T Z (n). We also formulate the lemma, which is somewhat more general than Theorem 2, and is of independent interest. Lemma 2. Let > 0. Then (K) = 0 and K T 1 (i 1)! (n) = · ( ) i 1 n! j=n+1 j + i=n+1 532 for n {K 1, . . . , 1, 0}. (17) 3. The problem of WoV Consider the multiple analog of the process of birth and death with linear growth, discussed in the previous section. As is known, birth and death processes can serve as primary stochastic models for describing changes in the size of a simple population of individuals, cells, bacteria or viruses. They are also models of simple queueing systems with possible call replication in queues. In some biological and computer systems, multiple cycles are observed, each of which consists of the following consecutive events and stages: the event of infection, the stage of the disease, the event of recovery, the stage of health. This model assumes that additional infections do not occur during the stage of the disease (as, for example, in the case of infections with the same strain of influenza virus). The initial number of viruses in each case of infection is considered random. In the case of a queueing system, the cycle includes an event of receiving a call packet into the system, a service stage with possible call replication, an event of reaching the zeroth queue, an idle system. It is also assumed that the system is immune to new call packages until the current service package is fully serviced. This model also corresponds to production systems with stochastic execution of homogeneous operations. The initial number of production operations in the plan for each cycle is also considered random. Such a multistage process can be interpreted as follows. Consider a cycle with number n N. The system receives (n) viruses (or calls, in the case of a queuing system) at the stopping time (n 1). In this case, the first cycle starts with zero time: (0) = 0. After the expiration of the time (n), the number of [ viruses reaches ) zero. That is, there are viruses in the system on the time interval (n 1), (n) , where (n) = (n 1) + (n). This interval is called the vulnerability window (or the window of vulnerability, abbreviated to WoV ), because the system is infected and therefore vulnerable. For example, for a living system, susceptibility to other diseases or threats increases, and the computer system becomes vulnerable, since there is a threat to its information security. During the time (n), that is, on the interval [(n), (n)), the system is free of viruses. At the stopping time (n) = (n) + (n) another infection occurs, that is, the next cycle begins. Thus, the difference between this scheme and the known process of birth and death with linear growth and immigration (see e.g., [20]) is that immigration is suspended for a time of non-zero process values. Apparently, it makes sense to call this process of immigration an episodic immigration. Below we present a formal mathematical description of this process. Let Y = (Y t ) t>0 be the process of birth and death with linear growth and episodic immigration, defined on the stochastic basis B. Suppose that Y can be represented in the form Y = G+AB, where A = (A t ) t>0 and B = (B t ) t>0 are the counting processes of the number of positive and negative jumps, Y t = Y t Y t , provided that Y (t) > 1 for t > 0: A t = I{Y s = 1, Y s > 1}, B t = I{Y s = 1, Y s > 1}, 0<s6t 0<s6t respectively. The process of episodic immigration G = (G t ) t>0 is defined as follows. Let N = {(n)} n=0,1,... be the set of independent identically distributed square-integrable random variables with values (n) N and the distribution { ( ) } P = p(k) = P (n) = k , k N for all n N 0 . 533 Suppose also that (0) is F 0 -measurable. Consider on B a Poisson process = ( t ) t>0 with a parameter > 0. Suppose that and N are independent. Let g = (g t ) t>0 be the counting process of the number of jumps of the process of episodic immigration, and G = (G t ) t>0 is the process of episodic immigration: t g t = I{Y s = 0} d s and G t = (0) + 0 g t (n). n=1 For t in WoV, Y = (Y t ) t>0 behaves similarly to the process X discussed in the Section 1. Thus, we assume that the compensators of the processes A and B on the stochastic basis B are the same as in (2), but with the corresponding replacement of X by Y : t t A t = · Y s ds and B t = · Y s ds. 0 0 Definition 6. We define the window of ) vulnerability (abbreviated as WoV ) on ( the interval [0, t] for any trajectory Y t () t>0 of the process Y as the random set W t Y () = {s [0, t] : Y s () > 0}, , t > 0. We also define the function L t Y = E {W t Y } for its Lebesgue measure {·}. Note that {W t Y ()} t = I{Y s > 1} ds = 0 l t Y (n). n=1 X k=1 L (k)·p(k) Assumption 3. Suppose that > 0 and < . From this assumption it follows that E (1) < and E (1) < and for all n N L X (k)·p(k). (18) E (n) = 1/ and E (n) = k=1 Theorem 3. For the WoV of Y, under Assumptions 2 and 3, {W t Y ()} L t Y = lim t>+ t>+ t t lim and lim t>+ L t Y t = L X (k)·p(k) k=1 1/ + k=1 where 534 L X (k) is given in Theorem 1. P-a.s. , L X (k)·p(k) 4. Proof of the results 4.1. Proof of Lemma 1 From (1) it follows that for any X 0 = K N, for all i N and t R + it holds t t I{X t > i} = I{X 0 > i} + I{X s = i 1} dA s I{X s = i} dB s . (19) 0 0 Since each integrable semimartingale (19) is the difference of submartingales, then t t I{X t > i} = I{K > i} + s . s I{X s = i} d B I{X s = i 1} d A 0 0 Therefore, for the compensators of the processes in (19), we obtain from (2) that i··l t X (i) + 1 if i = 1, X I{X t > i} = (20) (i 1)··l t (i 1) i··l t X (i) + 1 if 1 < i 6 K, X X (i 1)··l t (i 1) i··l t (i) if i > K. From (4), (5) and (20) we receive the following equalities: i·· X (i) + 1 = 0, if i = 1, X if 1 < i 6 K, (i 1)·· (i 1) i·· X (i) + 1 = 0, if i > K, (i 1)·· X (i 1) i·· X (i) = 0. (21) For = 0, the statement of Lemma 1 (and also Corollary 2) follows directly from (21). If > 0, then multiplying (21) by (/) (ni) and summing over i from 1 to n, we obtain for n N: { n (/) (ni) n · · X (n) = 0, if n 6 K, i=1 (22) K (ni) n · · X (n) = 0. if n > K, i=1 (/) The statement (12) of Lemma 1 follows from (22) with Notations 4. 2 4.2. Proof of Corollary 2 If = 0, then = 0. Hence, = and (13) coincides with (12). 2 4.3. Proof of Theorem 1 From (7) and Lemma 1 we receive K ( ( (nK) 1 n ) n ) L (K) = + = ·n ·n ·n ·n X n=1 1 · n=K+1 K n=1 1 1 · n n=1 n 1 (nK) + · . n n (23) n=K+1 If = 0, then = 0 and = . Therefore, from (23) we receive (9). If > 0, then > 0, and from (23) it follows that { } K 1 1 ( n ) 1 n L X (K) = · + + K · . (24) n n n n=1 n=1 n=K+1 The first term in braces in (24) is H(K), the second is log(1 ), and the third R K (). Thus, (8) is proved. 2 535 4.4. Proof of Corollary 1 If K = 1, then H(K) = 1. Therefore (11) obviously follows from (9). If > 0, 1 ·(1 ) = 1 , and for K = 1 we have then (10) follows from (8), since ( ) (/) K ·R K (/) = 1 + (/)· log 1 (/) . 2 4.5. Proof of Corollary 3 (n1)! K If = 1, then L Z (K) = E Z = T · K n=1 (n+1)! = T · K+1 . 2 4.6. Proof of Lemma 2 The equality (K) = 0 follows from Z 0 = K. From (14) we obtain the relation for the conditional distribution function F n (x) = P{l T Z (n) 6 x|F d(n) }, which is a simple generalization of the well-known Dellacherie theorem, [21]: dF n (x) = n · · I{x < T d(n)} dx. 1 F n (x) T d(n) x From this relation it follows that ( T d(n) x ) n F n (x) = 1 · I{x < T d(n)}. T d(n) T T d(n) x dF n (x) = , and Therefore, E{l T Z (n)|F d(n) } = n + 1 0 Z (n) = E{E(l T Z (n)|F d(n) )} = T (n) . n + 1 (25) n T Thus, from (16) and (25) we have the equality (n 1) = n+1 + (n) · n+1 , from which it easily follows that { } ( 1 { }) n (n 1) = · + (n) for all n K. (26) T n + 1/ n T { } Then we denote x(n) = T (n) · q(n), where the auxiliary function q(n) satisfies the recurrence relation q(n) n = with q(K) = 1. q(n 1) n + 1/ In this notation, (n) = T x(n) · for all n K 0 . q(n) (27) (28) Thus, from (26) and (27) we obtain the recurrence relation x(n 1) = q(n)/n + x(n) for all n K, with x(K) = 0. Hence, x(n) = K i=n+1 536 q(i)/i and q(n) = K n! (j + 1/) for all n < K K! j=n+1 (29) with x(K) = 0, q(K) = 1. Therefore, (17) follows from (28) and (29). 2 Since E Z 4.7. Proof of Theorem 2 = (0), (15) follows from (17) for n = 0. 2 4.8. Proof of Theorem 3 We define the auxiliary constant r = E (n) + E (n) = 1/ + (30) L X (k)·p(k) k=1 and the auxiliary function (t) = t/r · r, where · is the floor function. We note that lim t> (t)/t = 1 and for t > r Y ()} Y {W t Y ()} {W (t) (t) {W (t) ()} (t) {W t Y ()} = · + · . t (t) t (t) t (31) For the first summand in (31) we have ( ) [ ) Y {W t Y ()} {W (t) ()} /(t) 0, ((t) + 1)/(t) , and lim t> ((t) + 1)/(t) = 0 P-a.s. For the second summand we receive from the strong law of large numbers, lim t> Y ()} {W (t) (t) t/r 1 E (1) 1 = · lim · (i) = t> r t/r r P-a.s., i=1 and therefore the proof of the proposition follows from (18) and (30). 2 5. Discussion This article is devoted to the development of modeling methods for discrete stochastic systems based on semimartingale descriptions in terms of local time processes. For this it was necessary to formulate (and prove) a number of theoret- ical results for fairly simple systems. The proposed method is interesting not only because it allows obvious generalizations, but also by the possibilities of simple computer simulation. This is due primarily to the fact that all the descriptions and methods under consideration are trajectory. We also note that the definitions of JIT and WoV given in the paper are quite general and provide for generalizations of the results considered in the article.

About the authors

Aleksander Aleksandrovich Butov

Ulyanovsk State University

Email: butov.a.a@gmail.com
Doctor of physico-mathematical sciences, Professor 42, L. Tolstoy st., Ulyanovsk, 432017, Russian Federation

Anatoly A Kovalenko

Ulyanovsk State University

Email: anako09@mail.ru
42, L. Tolstoy st., Ulyanovsk, 432017, Russian Federation

References

Supplementary files