Ex. 2.4

The edge effect problem discussed on page 23 is not peculiar to uniform sampling from bounded domains. Consider inputs drawn from a spherical multinormal distribution $X \sim N (0, I_{p})$ . The squared distance from any sample point to the origin has a $χ_{p}^{2}$ distribution with mean $p$ . Consider a prediction point $x_{0}$ drawn from this distribution, and let $a = x_{0} / ‖ x_{0} ‖$ be an associated unit vector. Let $z_{i} = a^{T} x_{i}$ be the projection of each of the training points on this direction.

Show that the $z_{i}$ are distributed $N (0, 1)$ with expected squared distance from origin 1, while the target point has expected squared distance $p$ from the origin. Hence for $p = 10$ , a randomly drawn test point is about 3.1 standard deviations from the origin, while all the training points are on average one standard deviation along direction $a$ . So most prediction points see themselves as lying on the edge of the training set.

Soln. 2.4

Since $z_{i} = a^{T} x_{i}$ , $z_{i}$ is a linear combination of standard normal random variables, thus $z_{i}$ is normal. It's easy to see that $E [z_{i}] = 0$ and

Var (z_{i}) = ‖ a ‖^{2} Var (x_{i}) = Var (x_{i}) = 1.

There we know $z_{i} \sim N (0, 1)$ and the expected squared distance from origin is just its variance, which is 1. As for the target point $x_{t}$ , its squared distance to origin follows a $χ_{p}^{2}$ distribution and thus has mean $p$ .

For $p = 10$ , we have

sd (x_{t}) = \sqrt{Var (x_{t})} = \sqrt{10} \approx 3.16 .