Ex. 6.1

Show that the Nadaraya-Watson kernel smooth with fixed metric bandwidth \(\lambda\) and a Gaussian kernel is differentiable. What can be said for the Epanechnikov kernel? What can be said for the Epanechnikov kernel with adaptive nearest-neighbor bandwidth \(\lambda(x_0)\)?

Soln. 6.1

By definition of the Nadaraya-Watson kernel-weighted average, we have

\[\begin{equation} \hat f(x_0) = \frac{\sum_{i=1}^NK_\lambda(x_0, x_i)y_i}{\sum_{i=1}^NK_\lambda(x_0, x_i)}.\non \end{equation}\]

With Gaussian kernel

\[\begin{equation} K_\lambda(x_0, x) = \frac{1}{\sqrt {2\pi}\lambda}\exp \left( -\frac{(x-x_0)^2}{2\lambda^2} \right), \non \end{equation}\]

we know \(K_\lambda(x_0, x)\neq 0\) for all \(x_0\) and \(x\) in \(\mathbb{R}\), and is differentiable in \(x_0\), so that the Nadaraya-Watson kernel-weighted average is also differentiable in \(x_0\).

With Epanechnikov kernel

\[\begin{equation} K_\lambda(x_0, x) = D\left(\frac{|x-x_0|}{\lambda}\right),\non \end{equation}\]

with

\[\begin{equation} D(t) = \begin{cases} \frac{3}{4}(1-t^2), & \text{ if } |t|\le 1 \\ 0, & \text{ otherwise.} \end{cases}\non \end{equation}\]

Note that \(D(t)\) is continuous but not differentiable at \(t=1\), thus the kernel-weighted average holds the same property.

When the bandwidth is adaptive nearest-neighbor \(\lambda(x_0)\), \(\hat f(x_0)\) is still not differential by the same arguments when \(\frac{|x-x_0|}{\lambda(x_0)}\) approaches 1 from different directions.