Ex. 8.4
Ex. 8.4
Consider the bagging method of Section 8.7. Let our estimate \(\hat f(x)\) be the \(B\)-spline smoother \(\hat \mu(x)\) of Section 8.2.1. Consider the parametric bootstrap of equation (8.6), applied to this estimator. Show that if we bag \(\hat f(x)\), using the parametric bootstrap to generate the bootstrap samples, the bagging estimate \(\hat f_{\text{bag}}(x)\) converges to the original estimate \(\hat f(x)\) as \(B\ra\infty\).
Soln. 8.4
By definition of bagging we get
\[\begin{equation}
\label{eq:ex8-4bag}
\hat f_{\text{bag}}(x) = \frac{1}{B}\sum_{b=1}^B\hat f^\ast_b(x)
\end{equation}\]
where
\[\begin{equation}
\hat f^\ast_b(x) = Sy^\ast = S(\hat f(x) + \epsilon_b)\non
\end{equation}\]
\[\begin{equation}
S = N(N^TN)^{-1}N^T\non
\end{equation}\]
and \(\epsilon_b\sim N(0,\sigma^2)\) for \(B\)-spline smoother. Note that \(S^2 = S\), we obtain
\[\begin{equation}
\hat f^\ast_b(x) = S(\hat f(x) + \epsilon_b) = S(Sy+\epsilon_b) = Sy + S\epsilon_b.\non
\end{equation}\]
Therefore \(\eqref{eq:ex8-4bag}\) reduces to
\[\begin{equation}
\hat f_{\text{bag}}(x) = Sy + S\left(\frac{1}{B}\sum_{b=1}^B\epsilon_b\right).\non
\end{equation}\]
From here it's easy to see that
\[\begin{equation}
\lim_{B\ra\infty} \hat f_{\text{bag}}(x) = Sy = \hat f(x).\non
\end{equation}\]