Ex. 3.4

Ex. 3.4

Show how the vector of least squares coefficients can be obtained from a single pass of the Gram-Schmidt procedure (Algorithm 3.1). Represent your solution in terms of the QR decomposition of \(\textbf{X}\).

Soln. 3.4

Given the QR decomposition \(\textbf{X}=\textbf{Q}\textbf{R}\), we know that (see (3.32) in the text)

\[\begin{equation} \label{eq:3-4a} \hat\beta = \textbf{R}^{-1}\textbf{Q}^T\textbf{y}. \end{equation}\]

Since \(\textbf{Q}\) and \(\textbf{R}\) are computed from the Gram-Schmidt procedure on \(\textbf{X}\), as we are producing \(\textbf{q}_k\) for \(\textbf{Q}\), we can calculate \(\textbf{q}_k^T\textbf{y}\) so that \(\textbf{Q}^T\textbf{y}\) is sequentially filled. After \(\textbf{R}\) is computed, we can solve \(\textbf{R}^{-1}\) in a backward way since its upper triangular. Then we are able to compute \(\beta\) as given in \(\eqref{eq:3-4a}\).