Ex. 15.4

Ex. 15.4

Suppose $x_i,i=1,...,N$ are iid $(\mu ,{\sigma }^2)$. Let ${\overline{x}}_1^{\ast }$ and ${\overline{x}}_2^{\ast }$ be two bootstrap realizations of the sample mean. Show that the sampling correlation $\text{corr}({\overline{x}}_1^{\ast },{\overline{x}}_2^{\ast })=\frac{n}{2n-1}\approx 50\mathrm{\%}$. Along the way, derive $\text{var}({\overline{x}}_1^{\ast })$ and the variance of the bagged mean ${\overline{x}}_{\text{bag}}$. Here $\overline{x}$ is a linear statistic; bagging produces no reduction in variance for linear statistics.

Soln. 15.4

Denote

$$\begin{array}{r}{\overline{x}}_1^{\ast }=\frac{1}{n}\sum _{i=1}^n{\hat{x}}_i,{\overline{x}}_2^{\ast }=\frac{1}{n}\sum _{i=1}^n{\tilde{x}}_i,\end{array}$$

where $\{{\hat{x}}_i,i=1,...,n\}$ and $\{{\tilde{x}}_i,i=1,...,n\}$ are realizations from the first and the second bootstrap respectively.

Note that both ${\hat{x}}_i$ and ${\tilde{x}}_i$ are sampled from the empirical distribution of $\{x_i,i=1,...,n\}$.

Therefore, for any $1\le i\le n$, we have

$$\begin{array}{rcl}E[{\hat{x}}_i]& =& E[{\tilde{x}}_i]=\mu ,\\ \text{var}({\hat{x}}_i)& =& \text{var}({\tilde{x}}_i)={\sigma }^2.\end{array}$$

Also, for any $1\le i,j\le n$, we have

$$\begin{array}{r}\text{cov}({\hat{x}}_i,{\tilde{x}}_j)=\frac{{\sigma }^2}{n}.\end{array}$$

Then, we know

$$\begin{array}{r}\text{cov}({\overline{x}}_1^{\ast },{\overline{x}}_2^{\ast })=\frac{1}{n^2}(\sum _{i,j}^n\text{cov}({\hat{x}}_i,{\tilde{x}}_j))=\frac{{\sigma }^2}{n},\end{array}$$

and

$$\begin{array}{rcl}\text{var}({\overline{x}}_1^{\ast })& =& \text{var}\left(\frac{1}{n}\sum _{i=1}^n{\tilde{x}}_i\right)\\ & =& \frac{1}{n^2}(\sum _{i=1}^n\text{var}(x_i)+\sum _{j\ne k}^n\text{cov}({\hat{x}}_j,{\hat{x}}_k))\\ & =& \frac{1}{n^2}(n\cdot {\sigma }^2+(n^2-n)\cdot \frac{{\sigma }^2}{n})\\ & =& \frac{(2n-1){\sigma }^2}{n^2}.\end{array}$$

Therefore we know that

$$\text{corr}({\overline{x}}_1^{\ast },{\overline{x}}_2^{\ast })=\frac{\text{cov}({\overline{x}}_1^{\ast },{\overline{x}}_2^{\ast })}{\sqrt{\text{var}({\overline{x}}_1^{\ast })\text{var}({\overline{x}}_2^{\ast })}}=\frac{n}{2n-1}.$$

We have already derived $\text{var}({\overline{x}}_1^{\ast })$ above. For ${\overline{x}}_{\text{bag}}$, assume we have $B$ realizations, then

$$\begin{array}{rcl}\text{var}({\overline{x}}_{\text{bag}})& =& \text{var}\left(\frac{1}{B}\sum _{i=1}^B{\overline{x}}_i^{\ast }\right)\\ & =& \frac{1}{B^2}\sum _{i=1}^B\text{var}({\overline{x}}_i^{\ast })+\frac{1}{B^2}\sum _{j\ne k}^B\text{cov}({\overline{x}}_j^{\ast },{\overline{x}}_k^{\ast })\\ & =& \frac{1}{B}\cdot \frac{(2n-1){\sigma }^2}{n^2}+\frac{B-1}{B}\cdot \frac{{\sigma }^2}{n}\\ & =& \frac{(2n-1)+(B-1)n}{B{n}^2}\cdot {\sigma }^2.\end{array}$$