Ex. 3.24

Ex. 3.24

LAR directions. Using the notation around equation (3.55) on page 74, show that the LAR direction makes an equal angle with each of the predictors in ${\mathcal{A}}_k$.

Soln. 3.24

By definition of LAR we have

$$\begin{array}{rcl}{\mathbf{\text{X}}}_{{\mathcal{A}}_k}^T{u}_k& =& {\mathbf{\text{X}}}_{{\mathcal{A}}_k}^T({\mathbf{\text{X}}}_{{\mathcal{A}}_k}{\delta }_k)\\ & =& {\mathbf{\text{X}}}_{{\mathcal{A}}_k}^T{\mathbf{\text{X}}}_{{\mathcal{A}}_k}({\mathbf{\text{X}}}_{{\mathcal{A}}_k}^T{\mathbf{\text{X}}}_{{\mathcal{A}}_k}{)}^{-1}{\mathbf{\text{X}}}_{{\mathcal{A}}_k}^T{r}_k\\ & =& {\mathbf{\text{X}}}_{{\mathcal{A}}_k}^T{r}_k.\end{array}$$

By procedures of LAR, a new predictor $x_{j^{\prime }}$ is added when the absolute value of $x_{j^{\prime }}^{T}r$ equals that of $x_j^{T}r$ for all predictors $x_j\in {\mathcal{A}}_k$, we know the direction $u_k$ makes an equal angle with all predictors in ${\mathcal{A}}_k$.