next up previous
Next: Normal matrices and diagonalizability Up: algebra Previous: Eigenvalues and matrix diagonalization

Generalized eigenvalue problem

The generalized eigenvalue problem of two symmetric matrices ${\bf A}={\bf A}^T$ and ${\bf B}={\bf B}^T$ is to find a scalar $\lambda$ and the corresponding vector ${\bf\phi}$ for the following equation to hold:

\begin{displaymath}
{\bf A}{\bf\phi}_i=\lambda_i{\bf B}{\bf\phi}_i,
\;\;\;\;\;\;\;(i=1,\cdots,n)
\end{displaymath}

or in matrix form

\begin{displaymath}
{\bf A}{\bf\Phi}={\bf B}{\bf\Phi}{\bf\Lambda}
\end{displaymath}

The eigenvalue and eigenvector matrices ${\bf\Lambda}$ and ${\bf\Phi}$ can be found in the following steps.

The Rayleigh quotient of two symmetric matrices ${\bf A}$ and ${\bf B}$ is a function of a vector ${\bf w}$ defined as:

\begin{displaymath}
R({\bf w})=\frac{{\bf w}^T{\bf A}{\bf w}}{{\bf w}^T{\bf B}{\bf w}}
\end{displaymath}

To find the optimal ${\bf w}$ corresponding to the extremum (maximum or minimum) of $R({\bf w})$, we find its derivative with respect to ${\bf w}$:

\begin{displaymath}
\frac{d}{d{\bf w}}R({\bf w})
=\frac{2{\bf A}{\bf w}({\bf w}^...
...\bf w}({\bf w}^T{\bf A}{\bf w})}
{({\bf w}^T{\bf B}{\bf w})^2}
\end{displaymath}

Setting it to zero we get

\begin{displaymath}
{\bf A}{\bf w}({\bf w}^T{\bf B}{\bf w})={\bf B}{\bf w}({\bf...
...bf B}{\bf w}
=R({\bf w}){\bf B}{\bf w}=\lambda {\bf B}{\bf w}
\end{displaymath}

The second equation can be recognized as a generalized eigenvalue problem with $\lambda=R({\bf w})$ being the eigenvalue and and ${\bf w}$ the corresponding eigenvector. Solving this we get the vector ${\bf w}={\bf\phi}$ corresponding to the maximum/minimum eigenvalue $\lambda=R({\bf w})$, which maximizes/minimizes the Rayleigh quotient.


next up previous
Next: Normal matrices and diagonalizability Up: algebra Previous: Eigenvalues and matrix diagonalization
Ruye Wang 2014-09-01