Stochastic Processes

APPENDIX E

Stochastic Processes

For detailed treatments, see [B.12, B.17, B.18, B.21], and also [B.44, chapters 2 and 3], [B.49, Chapter 1]. An additional valuable reference is: J. Chandrasekhar, "Stochastic Problems in Physics and Astronomy," Rev. Mod. Phys. 15 (1943), 1-89.

E.1 GENERAL PROPERTIES Stochastic processes. Consider an ordinary function of time, x(t). Given a generic instant tl, x ( t l ) is a single number. 1 In a stochastic process, x(tl) is instead a random variable, which takes values in some interval (e.g., -oo < x < +oo) with probability density Pl(Xl,tl). In other words, P l ( X l , t l ) d x l is the probability of finding x(tl) in the interval Xl < x(tl) < Xl + dXl. x(t) is a r a n d o m variable for any value of t, w h i c h is s u m m a r i z e d b y saying that x(t) is a r a n d o m function of time. Separable stochastic processes. Consider the set of time instants t I <_ t 2 9 . . ~ t, and the corresponding set of r a n d o m variables x(tl), x(t2), . 9 . , x(tn). Let Pn(x n, t,; x n _ 1, tn - 1;" 9 9 ; Xl, tl)dXndXn - 1 9 9 9 dXl be the joint probability of h a v i n g x, < x ( t , ) < x , + d x , , x , _ 1 < x ( t , _ 1) < x,-1 + d x , _ 1. . . . , Xl < x(tl) < Xl + dXl. If the k n o w l e d g e of all possible joint

distributions for all time instants and all values of n is sufficient to define the process, then the process is said to be separable. All processes considered in this a p p e n d i x will be a s s u m e d to be separable. Joint probability density. T h e joint probability density P, previously defined

m u s t fulfill the following requirements: ix might also be a vector or a quantity of more complex tensorial character, but this possibility will not be considered.

519

520

APPENDIX E Stochastic Processes (i) Positivity: Pn >- 0 (ii) Normalization: f Pn dxn dx~ _ 1 9 9 9 d x l = 1 (iii) Compatibility: f P.(x,, t , ; . . . ; Xk, t k ; . . . ; Xl, tl) d x k = P, - 1 (x,, t , ; . . . ; Xk+l, t k + l " , X k _ I , t k _ l ; . . . ; Xl, tl)

C o n d i t i o n a l probability density. I n addition to joint probability densities, one can introduce conditional probability densities, P , _ k lk(X,, t,; . . . ; Xk + 1, tk + 1 I X k , t k ; " 9 9 ; X l , t l ) " P , - k lk represents the probability density of finding x n < x(tn) < x , + d x , , . . . , x k + 1 < X(tk + 1) < Xk + 1 + dXk + 1, a s s u m i n g that one k n o w s in advance that x(tk) = Xk, x ( t k _ 1) = X k - 1, 9 9 9 , x ( t l ) = Xl. This definition is equivalent to Pn - klk (Xn,tn, " 9 9 ;

Xk +

1,tk + 11 Xk, tk; " 9 9 ; Xl,tl)

(E.1)

_ Pn(xn, t n ; . 9 . ;x 1, tl) Pk(Xk, tk; . . . ;X 1, tl)

We shall consider in particular the conditional probability density P l l , - l(xn, t , I Xn - 1, t , _ 1; 9 9 9; Xl, tl), which will be simply indicated by the symbol P. S t a t i o n a r y processes. A stochastic process is stationary in the strict sense if P, is invariant with respect to a translation in time for any n, that is, P , ( x n, t , + 7;, X n

_

1,t,

_

1 if-

G.

9 9 ;

x l , t l + T)

(E.2)

= Pn(xn,tn; Xn - 1 , t n - 1;" 9 9 ;Xl,tl)

Stationarity is sometimes considered in a weaker form, where the validity of Eq. (E.2) is required only for one-time and two-time distributions. This m e a n s that one requires P1 to be i n d e p e n d e n t of time, 2 Pl(X), and P2 to d e p e n d on the time difference At = t 2 - t I only, 3 P2(x, At; Xo). We shall see later that strong stationarity is implied by w e a k stationarity in M a r k o v processes. M e a n , autocorrelation, p o w e r s p e c t r u m . Given the distribution Pl(X, t), the m e a n value, , of the process at time t is

= f x Pl(X,t) d x

(E.3)

Given the two-time distribution P2(x', t'; x, t), the m e a n value of the p r o d u c t x ( t ) x ( t ' ) , k n o w n as the autocorrelation function, R ( t , t ' ) , is given b y R(t,t') = = H

x ' x P2(x', t'; x,t) d x ' d x

(E.4)

2We shall also use the symbol Po(x) to refer to the stationary distribution attained after some transient situation. 3The origin of times can always be identified with t 1 and does not need to be explicitly indicated.

E.1 GENERAL PROPERTIES

521

In particular, R(t,t) = represents the total power of the process at time t. One can also introduce the autocovariance C(t,t'), defined as C(t,t') = R(t,t') - . The autocovariance is the autocorrelation of the process x(t) - . In a stationary process, the mean value is independent of time, , and the autocorrelation depends only on the time difference At = t' - t, R(At). R(At) is an even function of At. According to the Wiener-Khintchine theorem, the Fourier transform of R(At) gives the power spectrum (or power spectral density) S(co) of the process: S(co) =

i

oo

R(At) exp(-icoAt) dAt

(E.5)

--oo

S(co) is a real, even function of ta By taking the inverse Fourier transform of Eq. (E.5), one obtains R(At) =

~oo S(co) exp(icoAt) dco

(E.6)

--oo

In particular, by considering R(At = 0) •X2>

=

=

, one finds

S(co)

_oo

d____~ 2~r

(E.7)

which justifies the name of power spectrum given to S(co). In the comparison with experiments, one often considers not S(co), but 2S(co), with the convention that powers are given by integrals of S(co) over positive frequencies only. Ergodic properties. Integrals like Eq. (E.3) or Eq. (E.4) represent so-called statistical ensemble averages. One imagines having many identical realizations of the process, each characterized by its own path x(t;f~), where f~ varies over the statistical ensemble. The probability densities P1, P2, and so on measure the relative frequency with which certain properties are present in the statistical ensemble. However, given a single sample path x(t;f~), the variable x will randomly explore the phase space of the problem as time proceeds, and the question arises as to what extent time averages over individual sample paths can be able to reproduce statistical ensemble averages. An ergodic process is a process where, under appropriate conditions, time averages are equivalent to ensemble averages. In particular, consider a stationary process, and introduce the random variable

lfT

XT = ~

-T x(t) dt

(E.8)

By definition, the mean value of x T is equal to the mean value of the process, = . Under appropriate conditions, not particularly re-

APPENDIX E Stochastic Processes

522

strictive, it can be shown that the variance of XT fluctuations around the mean goes to zero in the limit T ~ oo. This implies that the mean value of the stochastic process can be obtained from the time average of the individual path x(t): = lim

x(t) dt

T --, oo

(E.9)

-T

Analogous expressions give the autocorrelation function and the power spectrum in terms of time averages: x(t)x(t + At)dt

(E.IO)

x(t) exp(-itat) dt

(E.11)

R(at) = lim T~oo

-T T

S(to) = lira 2~ f T-+oo

-T

Notice in particular that the power spectrum can be obtained from the Fourier transform of the sample function x(t), a fact commonly exploited in the experimental study of stochastic problems. Gaussian processes. A Gaussian process is a process where all Pn distributions are multivariate Gaussian distributions. In particular, in a stationary Gaussian process, the one-time and the two-time distributions are given by the expressions Pl(X) =

1 [ ( / - - )21 V,2cro 2 exp 2o 2

1

P2(x'At;x~ = 2 r

(E.13)

- r2

exp[ - ( x -

L

(E.12)

) 2 -

2 r ( x - )(x 0 - ) + (x0 202(1 - r 2)

)21

where a 2 is the variance of the process: 0 2 ~- < ( X -

)2~ > = -- < X > 2

(E.14)

and r is the correlation coefficient: R(at) - 2 r(at) = _ 2

(E.15)

Note that r is a function of At, and that Irl - 1 under all circumstances.

523

E.2 MARKOV PROCESSES

E.2 MARKOV PROCESSES Markov property. A Markov (or Markovian) process is a short-memory process, where future evolution depends on past history only through present conditions. In more precise mathematical terms, one has that, for any n, P(Xn,tnlXn - 1,tn - 1;" 9 9; Xl,tl) = P(xn,tnlXn

-

1, tn

-

1)

(E.16)

The conditional probability density depends only on the most recent condition imposed on x. The knowledge of the function P(xl,t I I x0,t0) governs the statistics of the process. In fact, given the joint distribution Pn - 1, the knowledge of P permits one to calculate Pn through the relation (see Eq. (E.1)) Pn(xn,tn; Xn - 1, t, _ 1;" 9 9; Xl,tl) =

(E.17)

P(x,, t, lx, _ 1, t, _ 1) Pn - l(Xn - 1,tn - 1;" 9 9 ;Xl, tl) Equation (E.17) permits one to construct in sequence all joint distributions. If the process is stationary in the weak sense, then P is a function of At - t 2 - t I only, P(x, At I x0), because P can be expressed in terms of P1 and P2 (Eq. E.1). Note that a Markov process that is stationary in the weak sense is also stationary in the strong sense, as a consequence of Eq. (E.17). Chapman-Kolmogorov equation. A n important relation can be derived by combining the definition of conditional probability density, Eq. (E.1), with the Markov property, Eq. (E.16). First, one can write P(XB, tglxl,tl) -- f P211(xB, tg;x2,t21xl,tl) dx2

(E.18)

= fP(xs, tslxa, ta;xl,tl)P(xa, talxl,tl) dx2 By applying the Markov property, one then obtains P(xs, t31xl,tl) -- f P(x3,tslx2,t2)P(x2,t21xl,tl) dx2

(E.19)

This relation is known as the Chapman-Kolmogorov equation, and expresses a general compatibility requirement to be satisfied by the conditional probability density of any Markov process. Continuous Markov processes. The sample path x(t) of a Markov process may exhibit three kinds of behavior: jumps, where x(t) changes discontinuously by a finite amount at certain discrete times; drift, where the mean value changes in time; diffusion, where the variance - 2 changes in time. If no jumps take place, then the sample paths x(t) are

524

APPENDIX E Stochastic Processes

continuous and one has a continuous Markov process. It can be shown that, with probability one, the sample paths are continuous functions of t if, for any e > 0, lim 1 f P(x + ax, t + Atlx, t ) dAx = 0 ate0 ~-t laxl>E

(E.20)

uniformly in x, t, and z~t.

Fokker-Planck equation. Under appropriate assumptions, the ChapmanKolmogorov equation of a continuous Markov process can be reduced to a partial differential equation, known as the Fokker-Planck equation. Let us assume that the process is such that

x, t =--faX P(x + Z~x,t + Atlx, t ) dAx = A(x,t)At + O((at) 2)

(E.21)

<(AX)2>x,t ~ f(Ax) 2 P(x Jr- z~x,t + Atlx, t) dAx = B2(x,t)at + O((At) 2) (E.22)

<(Ax)n>x,t ~ f (/kX) n P(x + ax, t + atlx, t) dax = O((At) 2)

n> 2

(E.23)

where < . . . > x , t represents conditional averages at the time t + At, under given x at the time t. Then it can be shown that Eq. (E.19) is equivalent to the following partial differential equation for P:

0 p(x, tlxo ) + -~x 0 [A(x,t)P(x, tlxo)] - -~ 1 ~x a22 [Ba(x,t)P(x, tlxo)] = 0 3~

(E.24)

A(x,t) describes how the mean value of the process deviates from the given value x at time t and is accordingly called the drift term. On the other hand, B2(x,t) describes how the path diffuses around" the given value x at time t and is called the diffusion term. When the drift and diffusion terms are both independent of time, that is, one has A(x) and B(x), the process is said to be homogeneous. In a homogeneous process the choice of the time origin is irrelevant and P is determined uniquely by the time elapsed after t = 0. P(x, tlxo) represents the conditional probability density of finding the value x at time t > 0, given the fact that the process is equal to x 0 at time 0. Accordingly, P(x, tlxo) is defined for positive times and must satisfy the initial condition P(x,t = 0Ix0) = 8(x - x0). The boundary conditions are instead to be specified case by case, depending on the particular process considered. Equation (E.24) should be called, more precisely, forward Fokker-Planck equation, because it describes the probability density P(x, tlxo) of finding the value x at the variable final time t, given the value x 0 at the fixed

E.3 WIENER, ORNSTEIN-UHLENBECK, wHrrE-NOISE PROCESSES

525

initial time t o = 0. However, it is also possible to write the so-called backward Fokker-Planck equation for the probability density P(xlxo, to) of finding the value x 0 at the variable initial time t 0, given the value x at the fixed final time t = 0. This equation takes the form 3t o

3 P(xlxo, to) + A(xo, to)-~x~ P(xlxo,t o) + ~ B2(xo'to )

P(xlxo, to) = 0

(E.25)

and should be solved for negative times under the final condition

P(xlxo, to = O) = 8(x - Xo). Note that, in a homogeneous process, P(x, tlxo) = P(xlx o, -t).

E.3 WIENER-LI~VY PROCESS, ORNSTEIN-UHLENBECK PROCESS, G A U S S I A N WHITE NOISE

Wiener-Lfvy process. Let us consider Eq. (E.24) in the case where A(x,t) = 0 and B(x,t) is constant. We imagine working with appropriate normalized quantifies for which this constant is simply B(x,t) = 1. The associated process is the (normalized) Wiener-L6vy process, also termed Wiener process, and will be denoted by w(t). w(t) obeys the equation

0__p(w, tlwo )

1

Ot

2

a2 cgW2

p(w, tlwo ) = 0

(E.26)

The solution satisfying the initial condition P(w, Olwo) = 8(w - Wo) is 1

P(w, tlwo) - ~

[ ( w - w 0 ) 2] exp L 9_t J

(E.27)

The properties of the Wiener process are best expressed in terms of the increment Aw = w - w 0 of the process in the time interval At: (i)

Successive

increments

Aw I

--

w I

-

Wo,

Aw 2 --

w 2 -

Wl,

.

.

. are

independent and are stationary, in the sense that the probability density of Aw depends on At only; (ii) = 0; (iii) = at. Because it is a continuous Markov process, the Wiener process has continuous paths. However, the paths are nowhere differentiable, as suggested by the previous property (iii), which shows that Aw --- (At) 1/2, so that A w / A t cannot tend to any finite limit when At ~ 0.

526

APPENDIX E Stochastic Processes

Ornstein-Uhlenbeck process. This process is the extension of the Wiener process to the case where A(x,t) is not zero, but proportional to x. By setting A(x,t) = - x / rand B2(x,t) = 2 o 2 / t i n Eq. (E.24), where rrepresents the characteristic time constant of the problem, one obtains the equation: o32

0__p(x, tlxo ) _ ~0 [xP(x, tlxo)]_ o2 -d-~x2 p(x, tlxo ) rot

= 0

(E.28)

The solution satisfying the initial condition P(x, Olxo) - 8(x - Xo) is

P(x,t'xo) = 1V'2 era2exp ( - x2-~ 2 )k=~02-~. 1 Hk ( X 0 ) ( c rHk Y2 o'V~ =

) exp ( - - ~ ) (E.29)

1 exp{-[x-x~ V'2 ~ra2[1 - e x p ( - 2t/T)I 2o211 - e x p ( - 2t/T)I

where Hk(X ) is the Hermite polynomial of degree k. Note that Eq. (E.29) reduces to Eq. (E.27) when t < < r and 0 2 / r = 1/2. The first stages of the Omstein-Uhlenbeck process are identical to the Wiener process. In fact, the Wiener process is recovered from the Omstein-Uhlenbeck process by taking the limit r --~ o% a 2 --~ o% under constant ratio o 2 / r - 1/2. When t > > r, the memory of the initial condition x = x 0 at t = 0 is lost, and the Ornstein-Uhlenbeck process becomes stationary. In the stationary limit, the one-time probability distribution, obtained by taking the limit t --~ oo in Eq. (E.29), is equal to P0(x) =

1 (' X2 ) V,2cra2 exp - ~-~

(E.30)

Equation (E.30) shows that the stationary mean value is zero, = 0. The stationary autocorrelation R(At) can be calculated from Eq. (E.4), taking into account that P2(x',At;x) = P(x',Atlx)Po(x), and by considering that x and x' are proportional to the Hermite polynomials Hl(X ) and Hl(X' ), and are therefore orthogonal to all terms of the series of Eq. (E.29) except the one with k = 1. One finds R(At) = o2 exp(-[At[/~-)

(E.31)

Thus the power spectrum is Lorentzian (Eq. E.5): 2a2r S(w) = 1 + apt 2

(E.32)

Doob's theorem. A stationary Gaussian process is Markovian if and only if its covariance is exponential, that is, C(At) ~ exp(-Ihtl/r). In other

E.4 STOCHASTIC DIFFERENTIAL EQUATIONS

527

words, the only possible stationary, Gaussian, and Markovian process is the Ornstein-Uhlenbeck process.

Gaussian white noise. This is an idealized process that turns out to be useful under many circumstances in spite of the singular character of some of its properties. A possible way to introduce it is by taking an appropriate limit of the Ornstein-Uhlenbeck process. According to Eq. (E.31), the Omstein-Uhlenbeck process is characterized by the correlation time r. We consider the limit process where this correlation time is arbitrarily small, ~"~ 0. In order to prevent the process from just vanishing, we simultaneously take the limit 02 ~ o% in such a way that the product a2~" remains finite, say cr2~" = 1/2. This limit process is called Gaussian white noise, and will be denoted by n(t). According to Eq. (E.31) and Eq. (E.32), it has a singular autocorrelation R(At) = 8(At)

(E.33)

S(~) = 1

(E.34)

and a fiat (white) spectrum:

The singular character of the process appears clearly from the fact that its power is infinite. However, n(t) turns out to be extremely useful in summarizing the basic features of any process with correlation time negligibly short with respect to the characteristic time scale of the problem. Another important property is represented by the fact that the integral of n(t) is the Wiener process. In fact, one can show that t

f on(t ) dt' = w(t) - Wo

(E.35)

where w(t) is the normalized Wiener process characterized by the properties (i)-(iii) given in the previous paragraph. The result given by Eq. (E.35) can be expressed in a different form by writing the white noise process as n(t) = dw(t)/dt, that is, as the formal derivative of the Wiener process. We recall that the Wiener process is actually not differentiable at any point in the usual sense, which may be seen as another indication of the highly idealized and singular character of the white noise process.

E.4 S T O C H A S T I C D I F F E R E N T I A L E Q U A T I O N S

Langevin equation. The best example of stochastic differential equation in physics is represented by the Langevin equation describing Brownian

528

APPENDIX E Stochastic Processes

motion. In order to describe the random motion of a Brownian particle, one writes the following equation for the particle velocity v: dv -- = dt

v + K n(t); 7

n(t) =

dw dt

(E.36)

in which the particle acceleration dv/dt is equated to the total force acting on the particle. The central idea is that the force consists of contributions acting on very different scales that can be separated. The term -v/~" represents the macroscopic viscous force exerted by the medium. The fact that, on the microscopic scale, this damping results from a lot of tiny interactions between the Brownian particle and the molecules of the medium is then taken into account by adding a fluctuating part to the macroscopic average force. This microscopic random force is expected to undergo substantial variations in very short times and is accordingly described by the white-noise process n(t). As a differential equation, Eq. (E.36) is not well defined, because it contains the wildly oscillating random process n(t), characterized by singular properties. This aspect can be given a rigorous formulation in the frame of Ito calculus, discussed next, where it turns out that Eq. (E.36) describes a continuous Markov process. It is possible then to make use of the results expressed by Eq. (E.21)-Eq. (E.24) to derive the Fokker-Planck equation associated with the process. Given the short time interval At, we know from Eq. (E.35) that n(t)At = Aw(t), so that, to first order in At, v v, t -

--

At T

+ Kv, t

v =

(E.37)

At

T

2 = K 2 v,t

= K 2 At

(E.38)

v,t

where it has been taken account of the properties of the Wiener process and of the fact that Aw = w(t + At) - w(t) is independent of v(t). By comparing these equations with Eq. (E.22) and Eq. (E.23), we conclude that the Fokker-Planck equation of the process is characterized by A(v,t) = -v/~', B(v,t) = K, that is, v(t) coincides with the Ornstein-Uhlenbeck process. The value of the constant K is determined by imposing that the spontaneous velocity fluctuations in thermodynamic equilibrium be consistent with the theorem of equipartition of energy.

Ito stochastic differential equations. The difficulties mentioned in connection with the Langevin equation, Eq. (E.36), can be dealt with by introducing appropriate mathematical methods to treat stochastic differential equa-

E.4 STOCHASTIC DIFFERENTIAL EQUATIONS

529

tions, that is, differential equations in which some of the terms are stochastic processes. In particular, an Ito stochastic differential equation is an equation that, written in terms of differentials, takes the form

dx = A(x,t)dt + B(x,t)dw

(E.39)

where dw is the infinitesimal increment of the normalized Wiener process w(t). In the frame of Ito calculus, one shows that Eq. (E.39) describes a continuous Markov process obeying the Fokker-Planck equation given by Eq. (E.24), whose drift and diffusion coefficients, A(x,t) and B(x,t), precisely coincide with the quantities appearing in Eq. (E.39). Note that one might heuristically think of interpreting Eq. (E.39) as a sort of generalized Langevin equation of the form

dx = A(x,t) + B(x,t) dw dt dt

(E.40)

where the white-noise term is instantaneously weighted by B(x,t). This interpretation, apart from the problem of dealing with the singular process dw/dt, contains a serious ambiguity. In fact, white noise is a wildly fluctuating process, which may lead to sudden variations of x, so that the precise value of x to be used in the coefficient B(x,t) in front of dw/dt is undetermined, and Eq. (E.40), in the form in which it is written, has no precise meaning. Ito calculus is equivalent to giving a definite prescription to resolve this ambiguity, and it is only by accepting this prescription that Eq. (E.39) is shown to be equivalent to Eq. (E.24). However, the Ito interpretation is not the only one possible. In particular, the Stratonovich interpretation exists, which leads not to Eq. (E.24), but to the equation

a p(x, tlxo) + -~x a [A(x,t)P(x, tlxo) ] _ 2-~x 1 a B(x,t) ~x [B(x,t) P(x, tlxo) ] = 0 O~

(E.41)

All stochastic differential equations considered in this book are interpreted according to the Ito prescription.

Brownian motion in a potential. Consider a particle of position x, subject to a viscouslike macroscopic force F(x)and to microscopic random forces describing thermal agitation. We assume the motion to be overdamped, with the particle velocity instantaneously proportional at any time to the total force acting on it. The motion is then described by the Ito equation ydx = F(x)dt + V'2"ykBT dw

(E.42)

where ~, is the friction constant, k B is the Boltzmann constant, and T is the absolute temperature. The coefficient in front of dw is chosen so as to correctly describe thermodynamic equilibrium properties, according to

530

APPENDIX E Stochastic Processes

the so-called fluctuation-dissipation theorem. The corresponding FokkerPlanck equation (Eq. E.24) is a2

3'-~ap(x, tlxo ) + ~0 [F(x)P(x, tlxo)] - kBT---2 P(x, tlxo) = 0

(E.43)

This equation is known as the Smoluchowski equation.

E.5 FIRST-PASSAGE TIME This denomination, as well as others, like level-crossing time or exit time, summarizes a class of problems where one is interested in determining the statistical distribution of the time at which a given process crosses a certain level for the first time or, for processes evolving in multidimensional spaces, the time at which the process generically leaves a certain domain of the phase space by crossing its boundary. The first-passage time is a random variable, because different realizations of the process will reach the boundary at different times. We briefly discuss some methods that can be employed in the study of its statistical properties, and in particular of its average value (mean exit time).

First-passage time in homogeneous Markov processes. We limit the discussion to a one-dimensional and homogeneous Markov process described by the Fokker-Planck equation (see Eq. (E.24)): 1

2

-~P(x, tlxo) + O[A(x)P(x,t[xo)] - ~ ~x2[B2(x)P(x,t[xo)] = 0

(E.44)

where A(x) and B(x) are independent of time because of the assumed homogeneity of the process. It is convenient to write Eq. (E.44) as a continuity equation:

OP + oJ = 0 at 3x

(E.45)

3 [B2(x)P(x,tlxo)] J(x't[x~ = A(x)P(x't[x~ - 21 -~x

(E.46)

where

is called the probability current. Equation (E.45) shows that a variation in the probability P(x, tlx0)dx of occupying a state in the interval (x,x + dx) can be interpreted as the result of a flow of probability into or out of that interval. Let us now select a certain interval (a,b) of the x axis. The process

E.5 FIRST-PASSAGE TIME

531

starts, at t = 0, from a point x 0 inside this interval, a < x0 < b. We are interested in estimating the time at which the process will reach for the first time one of the boundaries x = a or x = b. The problem can be studied by solving Eq. (E.44) under the usual initial condition, namely P(x,t = 0[x0) = 8(x - x0), and under the assumption of absorbing boundary conditions at x = a and x = b:

P(a,tlxo) = P(b,t[xo) = 0

(E.47)

The absorbing boundaries guarantee that, once the process has reached one of the boundaries, it is removed from the (a, b) interval and no longer contributes to the statistics of the problem. On the other hand, they give rise to a continuous flow of probability out of the interval, so that at large times P ~ 0 and J ~ 0 in the whole interval. The rate at which the process leaves the interval (a,b) is obtained by integrating Eq. (E.45) over x. One obtains

b

f P(x, tlxo)dx

l

= J(b, tlxo) - J(a, tlXo)

(E.48)

a

Equation (E.48) shows that the probability current at the boundaries is just proportional to the probability density that a first-passage event takes place at time t. Due to the homogeneous character of the process, P(x, tlxo) = P(x[x o, -t). By making use of this equality in Eq. (E.46), we see that the probability current obeys, as a function of x 0 and - t , the backward Fokker-Planck equation (Eq. (E.25)). In particular, for the probability currents appearing in Eq. (E.48), we have 0

O~ J(q'tlx~ - A(xo) ~

0

1

02

J(q, tlXo) - ~ B2(x0) ~

l(q, tlXo) = 0

(E.49)

where q stands for a or b. According to Eq. (E.48), the total probabilities, 7ra(Xo) and orb(x0), that the process leave the interval (a, b) through x = a or x = b are given by the expressions

7r~(Xo) =

-

f;

l(a,t'lXo) d t '

7rdXo) =

f" 0

l(b,t']xo) dt'

(E.50)

and saris@ the normalization condition 7ra(XO) -Jr- 7rb(Xo)---~ 1

(E.51)

By integrating Eq. (E.49) over time, one finds that both 1ra(Xo) and r satisfy the differential equation 1 Ba(x0)

2

d 2 7ra,b d 7ra,b if- A(xo) = 0 dx 2 dx o

(E.52)

APPENDIX E Stochastic Processes

532

with the b o u n d a r y conditions

r r

) = 1, ) = 0,

r

= 0

r

) = 1

(E.53)

The solution is

7I'a(XO)

xo

O(U) du

if;0

r

= ~-

O(U) du

(E.54)

where

[

qJ(u) = exp - 2

Ba(u )

The conditional probability densities P~aXit(tlXo) and Pebxit(tlXo) that the process leave the interval at time t, conditioned to the fact that we already k n o w that it leaves the interval through x = a or x = b, are given by P~aXit(tlxo)

=

-

J(a'tlx~

Pebxit(tlXo) =

Tl'a(Xo)

J(b't]x~

(E.56)

TI'b(Xo)

Accordingly, the m e a n value of the exit time through x = a or x = b is

taexit.(x0)x _ _ f ~ 1 7 6 t, G ( t , IX0) dt' = 0

t~xit(Xo)

=

t' Pb(t'lXo) dt' -

l

f~t,J(a,t,lxo) dt,

TI'a(XO)

1

b(Xo)

(E.57)

' J(b,t'lxo) dt'

By multiplying Eq. (E.49) by t and by integrating over t, one finds that the two m e a n exit times both obey the same differential equation: d2

1 B2(x0)

-~

Sa,b

dfa,b q_ ,ll.a,b(Xo ) = 0

dx 2 + A(xo) dx o

(E.58)

where

/a,b(X0)

"- " a,b [~,Xo ~ ~a,b ~exit kXO) [. \

(E.59)

and

fa,b(a) = fa,b(b) = 0

(E.60)

These b o u n d a r y conditions derive from Eq. (E.53) and from the fact that taexit (.l~\ = t~xit(b) = 0 by definition.

E.5 FIRST-PASSAGE TIME

533

First-passage time for the Wiener process with drift. As an example of application of the techniques just discussed, let us consider the process

t

x(t) = ~ + w(t)

(E.61)

where w(t) is the normalized Wiener process discussed in Section E.3 and cr is a positive constant. This process represents diffusion to which a steady drift in the positive x direction is superposed. Thus, we have a flow that tends to favor exit events through x = b, more than through x = a. The Fokker-Planck equation for the process is

1

1 32

~P(x, tlxo) + ~ ~xP(x, tlxo) - -~-d-~x2P(x,tlXo) 0

(E.62)

which is a particular case of Eq. (E.44), with A(x) = 1/2cr and B(x) = 1. For this particular case, Eq. (E.54) and Eq. (E.55) can be immediately integrated, giving

71"a(Xo) = ~1

[ e x p ( a - x~0 )

-- e x p ( a - b~ )

1

J

(E.63)

r176 where the normalization constant is equal to C = 1 - exp

a - b/ cr

(E.64)

We shall discuss the properties of the mean exit time through x = b. The case x = a can be analyzed by similar methods. The exit time is obtained by solving Eq. (E.58), which takes the form

d2f6 1 d~ dx 2 + - cr dx o + 2orb(X0) = 0

(E.65)

The solution is to be sought in the interval a - x 0 - b, with fb(a) = fb(b) = 0. By inserting "trb(Xo)(Eq. (E.63)) into Eq. (E.65), one obtains an equation that can be integrated by elementary methods. The resulting mean exit time, calculated by inserting the solution into Eq. (E.59) and by taking into account Eq. (E.63), is

t xit xo : 4 Ib a2octh(ba)2 xoa2o cth(X~

E66,

534

APPENDIX E Stochastic Processes

In particular, in the limit x 0 ~ a, Eq. (E.66) gives

tebxit(a)=4a;2Ib-a2crc t h / b - 2 aa )

11

(E.67)

When x 0 --~ a, the first-passage events through x = b become progressively more rare. However, the result expressed by Eq. (E.67) remains finite because it represents a property of the distribution of those rare events, once they have been extracted from the entire set of exit events.