- Email: [email protected]
Compatibility conditions for a set of conditional Gaussian distributions
Statistics & Probability Letters 42 (1999) 127 – 130
Compatibility conditions for a set of conditional Gaussian distributions Chuanhai Liu ∗ Bell Laboratories, Room 2C-262, 600 Mountain Avenue, Murray Hill, NJ 07974, USA Received August 1997
Abstract Suppose that a n-dimensional (random) vector X ∈ Rn is partitioned P into m sets of components as X = (X1 ; : : : ; Xm ), m where the dimension of Xi is ni , i.e., dim(Xi ) = ni ; for i = 1; : : : ; m and n = n; with m Gaussian conditionals i=1 i Xi |X−i ∼ N
i +
X
!
Ai; j Xj ; i; i
;
j6=i
where X−i = (X1 ; : : : ; Xi−1 ; Xi+1 ; : : : ; Xm ); dim(A0i; j ) = (ni ; nj ); dim( i; i ) = (ni ; ni ); and i; i ¿ 0; for i = 1; : : : ; m: We present a set of simple conditions on Ai; j and i; i for i; j = 1; : : : ; m and j 6= i under which the conditionals are compatible. The conditions are obtained based on the idea that essentially considers the inverse of the problem in terms of the compatibility c 1999 Elsevier of the likelihood functions of the “parameters” i ; Ai; j , and i; i s. An illustrative example is also provided. Science B.V. All rights reserved Keywords: Gaussian sweep; Spatial statistics
1. The problem We consider the following conditional normal distributions: components as X = Suppose that a n-dimensional (random) vector X ∈ Rn is partitioned into m sets of P m (X1 ; : : : ; Xm ), where the dimension of Xi is ni , i.e., dim(Xi ) = ni ; for i = 1; : : : ; m and i=1 ni = n; with m Gaussian conditionals X Ai; j Xj ; i; i ; (1) Xi |X−i ∼ N i + j6=i
∗ E-mail:
[email protected].
c 1999 Elsevier Science B.V. All rights reserved 0167-7152/99/$ – see front matter PII: S 0 1 6 7 - 7 1 5 2 ( 9 8 ) 0 0 1 8 2 - 5
128
C. Liu / Statistics & Probability Letters 42 (1999) 127 – 130
where X−i = (X1 ; : : : ; Xi−1 ; Xi+1 ; : : : ; Xm ); dim(Ai; j ) = (ni ; nj ); dim( i; i ) = (ni ; ni ); and i; i ¿ 0; for i = 1; : : : ; m: Speci cation of joint distributions via conditionals is generally convenient in terms of simplicity for mathematical formulation of our understanding of the world. The technique is thus frequently used in modeling complex phenomena of the nature, for example, in the context of spatial processes (e.g., Besag, 1974; Besag et al., 1995). As a consequence of the use of the technique, checking the compatibility 1 (Arnorld and Press, 1989) of the speci ed conditionals becomes a mathematical burden. Sucient conditions for compatibility in the Markov random- eld setting in the context of spatial processes are provided by the Hammersley-Cli ord theorem (Besag, 1974). Recently, Hobert and Casella (1995,1996) introduced the concept of functional compatibility 2 and studied its implication in the context of Markov chain Monte Carlo methods. Following Hobert and Casella (1995,1996), Meng (1996) suggested an interesting technique called de-conditioning, which, as the author claimed, is generally dicult to implement in practice. For the conditionals in Eq. (1), we give a set of simple compatibility conditions in the next section. In contrast to the above methods, the conditions are obtained based on the idea that essentially considers the inverse of the problem in terms of the compatibility conditions for the m likelihood functions of the parameters i ; Ai; j , and i; i s, each is obtained by conditioning further, i.e., conditioning on both X−i and Xi ; via the Gaussian sweep operator for i = 1; : : : ; m:
2. The compatibility conditions The following theorem provides a set of simple conditions on Ai; j and i; i for i; j = 1; : : : ; m and j 6= i under which the conditionals in (1) are compatible. Theorem 1. Let Bi; j = i;−1i Ai; j for j = 1; : : : ; i − 1; i + 1; : : : ; m and Bi; i = − i;−1i ; then the conditionals (1) are compatible if and only if (I) Bi;0 j = Bj; i for i; j = 1; : : : ; m and (II) ≡ (−Bi; j )(n×n) ¿0: Moreover, if they are compatible, then the joint distribution is X ∼ N(; −1 ); where
I
−A1;2
: : : −A1; m
−1
−A2; 1 I : : : −A2; m = .. .. .. .. . . . . I −Am; 1 −Am; 2 : : :
1 2 .. .
:
m
A proof of the theorem can be accomplished by conditioning further via the Gaussian sweep operator, which is given as follows:
1
A set of conditionals is called compatible if there exists a joint distribution that generates the conditional distributions. A set of conditionals is said to be functionally compatible if there exists a function that may not be necessarily integrable but can be served as a joint distribution generating the conditionals. 2
C. Liu / Statistics & Probability Letters 42 (1999) 127 – 130
Proof. For each i ∈ {1; : : : ; m}, we apply ∗ ∗ ∗ A0i; 1 ∗ .. ∗ ∗ ∗ . ∗ ∗ ∗ ∗ ∗ A0i; i−1 Ai; 1 : : : Ai; i−1 i; i Ai; i+1 ∗ ∗ ∗ A0i; i+1 ∗ .. ∗ ∗ ∗ . ∗ ∗
∗
A0i; m
∗ ∗
∗ ∗ Sweep ⇒ Bi; 1 Block(i; i) ∗ ∗ ∗
∗
the Gaussian sweep operator as follows: ∗ ∗ ∗ ∗ ∗ ∗ : : : Ai; m ∗ ∗ ∗ ∗ ∗ ∗
∗
Bi;0 1
∗
∗
∗
.. .
∗
∗
∗
Bi;0 i−1 ∗
∗
: : : Bi; i−1 Bi; i
Bi; i+1
∗
∗
Bi;0 i+1
∗
∗
.. .
∗
∗
Bi;0 m
∗
∗
129
∗
∗
∗
∗ ∗ ∗ ∗ : : : Bi; m : ∗ ∗ ∗ ∗ ∗ ∗
(2)
Note i;−1i can be understood as the general inverse i;−i if i; i is singular. Since the matrix on the left-hand side of Eq. (2) is the matrix obtained by applying the Gaussian sweep operator to the covariance matrix
of the joint distribution of X with the pivotal block matrix corresponding to the covariance matrix of X−i , the matrix on the right-hand side of Eq. (2) equals − −1 : Thus the necessary and sucient conditions on Ai; j and i; i for i; j = 1; : : : ; m and j 6= i under which the conditionals in Eq. (1) are compatible are: 1. Bi;0 j = Bj; i for i; j = 1; : : : ; m; and 2. ≡ (−Bi; j )(n×n) ¿0: If the conditionals in Eq. (1) are compatible, that is, conditions (1) and (2) are satis ed, it is easy to verify that the joint distribution of X can be written as X ∼ N(; −1 ): This completes the proof. 3. An example The following example 1 serves as an illustrative example of the use of Theorem 1. Example 1. Suppose that a set of normal conditionals is given as follows (Hobert and Casella, 1995): X xj ; 1 xi |x−i ∼ Npi j6=i
130
C. Liu / Statistics & Probability Letters 42 (1999) 127 – 130
for i = 1; : : : ; m; that is, ni = 1; Ai; j = pi and i; i = 1 for i; j = 1; : : : ; m and i 6= j: In this example, Bi; j = pi for j = 6 i and Bi; i = −1: From condition (1), we have pi0 (=pi ) = pj for i; j = 1; : : : ; m; which leads to p1 = p2 = · · · pm ≡ p: From condition (2), that is, 1 −p −p 1 D(m×m) ≡ .. .. . . −p −p
: : : −p
: : : −p .. .. ¿ 0; . . ::: 1
we need the constraint on p: −1 ¡ p ¡
1 ; m−1
which is a direct result from |D(k×k) | = (1 + p)k−1 (1 − (k − 1)p) for k = 1; : : : ; m: 4. A remark The compatibility conditions can also be applied to a subset of the normal conditional distributions {[Xi |X−i ]: i ∈ S ⊂{1; : : : ; m}} to check if the conditionals in the subset are conditionally compatible given {Xj : j ∈ {1; : : : ; m}\S}: This allows for exible ways of grouping variables for checking and modifying the compatibilities of the conditionals in a sequential fashion. References Arnorld, B.C., Press, S.J., 1989. Compatible conditional distributions. J. Amer. Statist. Assoc. 84, 152–156. Besag, J., 1974. Spatial interaction and the statistical analysis of lattice systems (with discussion). J. Roy. Statist. Soc. Ser. B 36, 192–236. Besag, J., Green, P., Higdon, D., Mengersen, K., 1995. Bayesian computation and stochastic systems (with discussion). Statist. Sci. 10, 3–66. Hobert, J.P., Casella, G., 1995. Functional compatibility, Markov chains and Gibbs sampling with improper posterior. Technical Report 481, Dept. of Statistics, University of Florida. Hobert, J.P., Casella, G., 1996. The e ect of improper priors in Gibbs sampling in hierarchical linear mixed models. J. Amer. Statist. Assoc. 91, 1461–1473. Meng, X.L., 1996. Recursive de-conditioning and conditional compatibility, Comments on “Statistical Inference and Monte Carlo Algorithms” by George Casella. Test 5, 310–318.
Compatibility conditions for a set of conditional Gaussian distributions Chuanhai Liu ∗ Bell Laboratories, Room 2C-262, 600 Mountain Avenue, Murray Hill, NJ 07974, USA Received August 1997
Abstract Suppose that a n-dimensional (random) vector X ∈ Rn is partitioned P into m sets of components as X = (X1 ; : : : ; Xm ), m where the dimension of Xi is ni , i.e., dim(Xi ) = ni ; for i = 1; : : : ; m and n = n; with m Gaussian conditionals i=1 i Xi |X−i ∼ N
i +
X
!
Ai; j Xj ; i; i
;
j6=i
where X−i = (X1 ; : : : ; Xi−1 ; Xi+1 ; : : : ; Xm ); dim(A0i; j ) = (ni ; nj ); dim( i; i ) = (ni ; ni ); and i; i ¿ 0; for i = 1; : : : ; m: We present a set of simple conditions on Ai; j and i; i for i; j = 1; : : : ; m and j 6= i under which the conditionals are compatible. The conditions are obtained based on the idea that essentially considers the inverse of the problem in terms of the compatibility c 1999 Elsevier of the likelihood functions of the “parameters” i ; Ai; j , and i; i s. An illustrative example is also provided. Science B.V. All rights reserved Keywords: Gaussian sweep; Spatial statistics
1. The problem We consider the following conditional normal distributions: components as X = Suppose that a n-dimensional (random) vector X ∈ Rn is partitioned into m sets of P m (X1 ; : : : ; Xm ), where the dimension of Xi is ni , i.e., dim(Xi ) = ni ; for i = 1; : : : ; m and i=1 ni = n; with m Gaussian conditionals X Ai; j Xj ; i; i ; (1) Xi |X−i ∼ N i + j6=i
∗ E-mail:
[email protected].
c 1999 Elsevier Science B.V. All rights reserved 0167-7152/99/$ – see front matter PII: S 0 1 6 7 - 7 1 5 2 ( 9 8 ) 0 0 1 8 2 - 5
128
C. Liu / Statistics & Probability Letters 42 (1999) 127 – 130
where X−i = (X1 ; : : : ; Xi−1 ; Xi+1 ; : : : ; Xm ); dim(Ai; j ) = (ni ; nj ); dim( i; i ) = (ni ; ni ); and i; i ¿ 0; for i = 1; : : : ; m: Speci cation of joint distributions via conditionals is generally convenient in terms of simplicity for mathematical formulation of our understanding of the world. The technique is thus frequently used in modeling complex phenomena of the nature, for example, in the context of spatial processes (e.g., Besag, 1974; Besag et al., 1995). As a consequence of the use of the technique, checking the compatibility 1 (Arnorld and Press, 1989) of the speci ed conditionals becomes a mathematical burden. Sucient conditions for compatibility in the Markov random- eld setting in the context of spatial processes are provided by the Hammersley-Cli ord theorem (Besag, 1974). Recently, Hobert and Casella (1995,1996) introduced the concept of functional compatibility 2 and studied its implication in the context of Markov chain Monte Carlo methods. Following Hobert and Casella (1995,1996), Meng (1996) suggested an interesting technique called de-conditioning, which, as the author claimed, is generally dicult to implement in practice. For the conditionals in Eq. (1), we give a set of simple compatibility conditions in the next section. In contrast to the above methods, the conditions are obtained based on the idea that essentially considers the inverse of the problem in terms of the compatibility conditions for the m likelihood functions of the parameters i ; Ai; j , and i; i s, each is obtained by conditioning further, i.e., conditioning on both X−i and Xi ; via the Gaussian sweep operator for i = 1; : : : ; m:
2. The compatibility conditions The following theorem provides a set of simple conditions on Ai; j and i; i for i; j = 1; : : : ; m and j 6= i under which the conditionals in (1) are compatible. Theorem 1. Let Bi; j = i;−1i Ai; j for j = 1; : : : ; i − 1; i + 1; : : : ; m and Bi; i = − i;−1i ; then the conditionals (1) are compatible if and only if (I) Bi;0 j = Bj; i for i; j = 1; : : : ; m and (II) ≡ (−Bi; j )(n×n) ¿0: Moreover, if they are compatible, then the joint distribution is X ∼ N(; −1 ); where
I
−A1;2
: : : −A1; m
−1
−A2; 1 I : : : −A2; m = .. .. .. .. . . . . I −Am; 1 −Am; 2 : : :
1 2 .. .
:
m
A proof of the theorem can be accomplished by conditioning further via the Gaussian sweep operator, which is given as follows:
1
A set of conditionals is called compatible if there exists a joint distribution that generates the conditional distributions. A set of conditionals is said to be functionally compatible if there exists a function that may not be necessarily integrable but can be served as a joint distribution generating the conditionals. 2
C. Liu / Statistics & Probability Letters 42 (1999) 127 – 130
Proof. For each i ∈ {1; : : : ; m}, we apply ∗ ∗ ∗ A0i; 1 ∗ .. ∗ ∗ ∗ . ∗ ∗ ∗ ∗ ∗ A0i; i−1 Ai; 1 : : : Ai; i−1 i; i Ai; i+1 ∗ ∗ ∗ A0i; i+1 ∗ .. ∗ ∗ ∗ . ∗ ∗
∗
A0i; m
∗ ∗
∗ ∗ Sweep ⇒ Bi; 1 Block(i; i) ∗ ∗ ∗
∗
the Gaussian sweep operator as follows: ∗ ∗ ∗ ∗ ∗ ∗ : : : Ai; m ∗ ∗ ∗ ∗ ∗ ∗
∗
Bi;0 1
∗
∗
∗
.. .
∗
∗
∗
Bi;0 i−1 ∗
∗
: : : Bi; i−1 Bi; i
Bi; i+1
∗
∗
Bi;0 i+1
∗
∗
.. .
∗
∗
Bi;0 m
∗
∗
129
∗
∗
∗
∗ ∗ ∗ ∗ : : : Bi; m : ∗ ∗ ∗ ∗ ∗ ∗
(2)
Note i;−1i can be understood as the general inverse i;−i if i; i is singular. Since the matrix on the left-hand side of Eq. (2) is the matrix obtained by applying the Gaussian sweep operator to the covariance matrix
of the joint distribution of X with the pivotal block matrix corresponding to the covariance matrix of X−i , the matrix on the right-hand side of Eq. (2) equals − −1 : Thus the necessary and sucient conditions on Ai; j and i; i for i; j = 1; : : : ; m and j 6= i under which the conditionals in Eq. (1) are compatible are: 1. Bi;0 j = Bj; i for i; j = 1; : : : ; m; and 2. ≡ (−Bi; j )(n×n) ¿0: If the conditionals in Eq. (1) are compatible, that is, conditions (1) and (2) are satis ed, it is easy to verify that the joint distribution of X can be written as X ∼ N(; −1 ): This completes the proof. 3. An example The following example 1 serves as an illustrative example of the use of Theorem 1. Example 1. Suppose that a set of normal conditionals is given as follows (Hobert and Casella, 1995): X xj ; 1 xi |x−i ∼ Npi j6=i
130
C. Liu / Statistics & Probability Letters 42 (1999) 127 – 130
for i = 1; : : : ; m; that is, ni = 1; Ai; j = pi and i; i = 1 for i; j = 1; : : : ; m and i 6= j: In this example, Bi; j = pi for j = 6 i and Bi; i = −1: From condition (1), we have pi0 (=pi ) = pj for i; j = 1; : : : ; m; which leads to p1 = p2 = · · · pm ≡ p: From condition (2), that is, 1 −p −p 1 D(m×m) ≡ .. .. . . −p −p
: : : −p
: : : −p .. .. ¿ 0; . . ::: 1
we need the constraint on p: −1 ¡ p ¡
1 ; m−1
which is a direct result from |D(k×k) | = (1 + p)k−1 (1 − (k − 1)p) for k = 1; : : : ; m: 4. A remark The compatibility conditions can also be applied to a subset of the normal conditional distributions {[Xi |X−i ]: i ∈ S ⊂{1; : : : ; m}} to check if the conditionals in the subset are conditionally compatible given {Xj : j ∈ {1; : : : ; m}\S}: This allows for exible ways of grouping variables for checking and modifying the compatibilities of the conditionals in a sequential fashion. References Arnorld, B.C., Press, S.J., 1989. Compatible conditional distributions. J. Amer. Statist. Assoc. 84, 152–156. Besag, J., 1974. Spatial interaction and the statistical analysis of lattice systems (with discussion). J. Roy. Statist. Soc. Ser. B 36, 192–236. Besag, J., Green, P., Higdon, D., Mengersen, K., 1995. Bayesian computation and stochastic systems (with discussion). Statist. Sci. 10, 3–66. Hobert, J.P., Casella, G., 1995. Functional compatibility, Markov chains and Gibbs sampling with improper posterior. Technical Report 481, Dept. of Statistics, University of Florida. Hobert, J.P., Casella, G., 1996. The e ect of improper priors in Gibbs sampling in hierarchical linear mixed models. J. Amer. Statist. Assoc. 91, 1461–1473. Meng, X.L., 1996. Recursive de-conditioning and conditional compatibility, Comments on “Statistical Inference and Monte Carlo Algorithms” by George Casella. Test 5, 310–318.