Ex. 14.7
Ex. 14.7
Derive (14.51) and (14.52) in Section 14.5.1. Show that \(\hat\mu\) is not unique, and characterize the family of equivalent solutions.
Soln. 14.7
We need to miminize the reconstruction error
\[\begin{equation}
\min_{\mu, \{\lambda_i\}, \bb{V}_q} \sum_{i=1}^N\|x_i-\mu-\bb{V}_q\lambda_i\|^2.\non
\end{equation}\]
Taking derivatives w.r.t to \(\mu\) and \(\lambda_i\) and setting them to zero, we get
\[\begin{eqnarray}
&&\sum_{i=1}^N(x_i-\mu-\bb{V}_q\lambda_i) = 0\non\\
&&\bb{V}_q^T(x_i-\mu-\bb{V}_q\lambda_i) = 0.\non
\end{eqnarray}\]
Since \(\bb{V}_q^T\bb{V}_q = \bb{I}\), from the condition on \(\lambda_i\) we have
\[\begin{equation}
\lambda_i = \bb{V}_q^T(x_i-\mu),\non
\end{equation}\]
and we plug this into the condition for \(\mu\) and we get
\[\begin{equation}
(\bb{I} - \bb{V}_q\bb{V}_q^T)\sum_{i=1}^N(x_i-\mu) = 0.\non
\end{equation}\]
Therefore, we see that
\[\begin{eqnarray}
\hat\mu &=& \bar x\non\\
\hat\lambda_i &=&\bb{V}_q^T(x_i-\bar x)\non
\end{eqnarray}\]
is a set of optimized solutions, however not unique. The family of equivalent solutions is characterized by the set of \(\hat\mu\) that yields \(\sum_{i=1}^N(x_i-\hat\mu)\) lying in the null space of \((\bb{I} - \bb{V}_q\bb{V}_q^T)\).