Ex. 4.1

Ex. 4.1

Show how to solve the generalized eigenvalue problem \(a^T\textbf{B}a\) subject to \(a^T\textbf{W}a = 1\) by transforming to a standard eigenvalue problem.

Soln. 4.1

We are solving a constraint optimization problem

\[\begin{eqnarray} &&\max_{a} a^T\textbf{B}a\nonumber\\ \text{s.t.} && a^T\textbf{W}a = 1.\nonumber \end{eqnarray}\]

The Lagrangian multiplier is

\[\begin{equation} L(a,\lambda) = a^T\textbf{B}a - \lambda(a^T\textbf{W}a-1).\nonumber \end{equation}\]

Taking partial derivative w.r.t \(a\) and setting it to be zero we get

\[\begin{equation} \frac{\partial L(a, \lambda)}{\partial a} = 2\textbf{B}a + \lambda(2\textbf{W}a)=0,\nonumber \end{equation}\]

which is equivalent to

\[\begin{equation} \label{eq:ex41eigen} \textbf{W}^{-1}\textbf{B}a = \lambda a. \end{equation}\]

Now it's easy to see that \(\eqref{eq:ex41eigen}\) is a standard eigenvalue problem.