Ex. 17.3

Let \(\bm{\Sigma}\) be the covariance matrix of a set of \(p\) variables \(X\). Consider the partial covariance matrix \(\bm{\Sigma}_{a.b} = \bm{\Sigma}_{aa} - \bm{\Sigma}_{aa} - \bm{\Sigma}_{ab}\bm{\Sigma}_{bb}^{-1}\bm{\Sigma}_{ba}\) between the two subsets of variables \(X_a = (X_1, X_2)\) consisting of the first two, and \(X_b\) the rest. This is the covariance matrix between these two variables, after linear adjustment for all the rest. In the Gaussian distribution, this is the covariance matrix of the conditional distribution of \(X_a|X_b\). The partial correlation coefficient \(\rho_{jk}|\text{rest}\) between the pair \(X_a\) conditional on the rest \(X_b\), is simply computed from this partial covariance. Define \(\bm{\Theta}=\bm{\Sigma}^{-1}\).

Show that \(\bm{\Sigma}_{a.b}=\bm{\Theta}_{aa}^{-1}\).
Show that if any off-diagonal element of \(\bm{\Theta}\) is zero, then the partial correlation coefficient between the corresponding variables is zero.
Show that if we treat \(\bm{\Theta}\) as if it were a covariance matrix, and compute the corresponding correlation matrix

\[\begin{equation} \bb{R} = \text{diag}(\bm{\Theta})^{-1/2}\cdot\bm{\Theta}\cdot \text{diag}(\bm{\Theta})^{-1/2},\non \end{equation}\]

then \(r_{jk} = -\rho_{jk}|\text{rest}\).

Soln. 17.3

(1) We write

\[\begin{equation} \bm{\Sigma} = \begin{pmatrix} \bm{\Sigma}_{aa} & \bm{\Sigma}_{ab}\\ \bm{\Sigma}_{ba} & \bm{\Sigma}_{bb} \end{pmatrix}.\non \end{equation}\]

Then the desired result \(\bm{\Sigma}_{a.b} = \bm{\Theta}_{aa}^{-1}\) follows directly from the block matrix inverse (e.g., see 8.4.3 in \cite{matbook}).

(2) Assume that \(\theta_{jk} = 0\) for some \(j < k\). Note \(X_a = (X_j, X_k)\) in such case. Then by results above we have

\[\begin{equation} \bm{\Sigma}_{a.b} = \bm{\Theta}_{aa}^{-1} = \begin{pmatrix} \theta_{jj}^{-1} & 0 \\ 0 & \theta_{kk}^{-1} \end{pmatrix}.\non \end{equation}\]

Therefore the partial correlation coefficient is

\[\begin{equation} \rho_{jk|\text{rest}} = \frac{\left(\bm{\Sigma}_{a.b}\right)_{12}}{\sqrt{\theta_{jj}^{-1}\theta_{kk}^{-1}}}=0.\non \end{equation}\]

(3) Following the notations above, we have

\[\begin{equation} \bm{\Sigma}_{a.b} = \bm{\Theta}_{aa}^{-1} = \begin{pmatrix} \theta_{jj} & \theta_{jk} \\ \theta_{kj} & \theta_{kk} \end{pmatrix}^{-1} = \frac{1}{\theta_{jj}\theta_{kk}-\theta_{jk}\theta_{kj}} \begin{pmatrix} \theta_{kk} & -\theta_{jk}\\ -\theta_{kj} & \theta_{jj} \end{pmatrix}.\non \end{equation}\]

Then we have

\[\begin{eqnarray} \rho_{jk|\text{rest}} &=& -\frac{\theta_{jk}}{\theta_{jj}\theta_{kk}-\theta_{jk}\theta_{kj}}\Bigg/\sqrt{\frac{\theta_{kk}}{\theta_{jj}\theta_{kk}-\theta_{jk}\theta_{kj}}\cdot\frac{\theta_{jj}}{\theta_{jj}\theta_{kk}-\theta_{jk}\theta_{kj}}}\non\\ &=&-\frac{\theta_{jk}}{\sqrt{\theta_{jj}\theta_{kk}}}\non\\ &=&-r_{jk},\non \end{eqnarray}\]

where the last equation follows from the definition of \(r_{jk}\).