Ex. 7.6
Ex. 7.6
Show that for an additive-error model, the effective degrees-of-freedom for the \(k\)-nearest-neighbors regression fit is \(N/k\).
Soln. 7.6
Note that for this \(k\)-nearest-neighbors model, it's a linear smoother. To see that, note
\[\begin{equation}
\hat Y(x) = \frac{1}{k}\sum_{i: x_i\in N_k(x)}y_i = \frac{1}{k}\sum_{i=1}^N\eta_i y_i\non
\end{equation}\]
where \(\eta_i = 1\) if \(x_i\in N_k(x)\) and 0 otherwise.
So we can write
\[\begin{equation}
\hat Y = \frac{1}{k}\bb{S}\bb{y}\non
\end{equation}\]
in which \(\bb{S}\) is a binary matrix with diagonal elements being 1 since the nearest one (itself) must be included in estimation. Therefore, the effective df is simply
\[\begin{equation}
\frac{1}{k}\text{trace}(\bb{S}) = \frac{N}{k}.\non
\end{equation}\]