Ex. 8.4

Ex. 8.4

Consider the bagging method of Section 8.7. Let our estimate $\hat{f}(x)$ be the $B$-spline smoother $\hat{\mu }(x)$ of Section 8.2.1. Consider the parametric bootstrap of equation (8.6), applied to this estimator. Show that if we bag $\hat{f}(x)$, using the parametric bootstrap to generate the bootstrap samples, the bagging estimate ${\hat{f}}_{\text{bag}}(x)$ converges to the original estimate $\hat{f}(x)$ as $B\to \mathrm{\infty }$.

Soln. 8.4

By definition of bagging we get

$$\begin{array}{}\text{(1)}& {\hat{f}}_{\text{bag}}(x)=\frac{1}{B}\sum _{b=1}^B{\hat{f}}_b^{\ast }(x)\end{array}$$

where

$${\hat{f}}_b^{\ast }(x)=S{y}^{\ast }=S(\hat{f}(x)+{ϵ}_b)$$

$$S=N(N^{T}N{)}^{-1}{N}^T$$

and ${ϵ}_b\sim N(0,{\sigma }^2)$ for $B$-spline smoother. Note that $S^2=S$, we obtain

$${\hat{f}}_b^{\ast }(x)=S(\hat{f}(x)+{ϵ}_b)=S(Sy+{ϵ}_b)=Sy+S{ϵ}_b.$$

Therefore $\text{(1)}$ reduces to

$${\hat{f}}_{\text{bag}}(x)=Sy+S\left(\frac{1}{B}\sum _{b=1}^B{ϵ}_b\right).$$

From here it's easy to see that

$$\underset{B\to \mathrm{\infty }}{lim}{\hat{f}}_{\text{bag}}(x)=Sy=\hat{f}(x).$$