Next: Normal matrices and diagonalizability Up: algebra Previous: Eigenvalues and matrix diagonalization

# Generalized eigenvalue problem

The generalized eigenvalue problem of two symmetric matrices and is to find a scalar and the corresponding vector for the following equation to hold:

or in matrix form

The eigenvalue and eigenvector matrices and can be found in the following steps.

• Solve the eigenvalue problem of to find its diagonal eigenvalue matrix and orthogonal eigenvector matrix so that

• Left and right multiplying both sides of the second equation above by (whitening) we get

We define

and get

Note that is not orthogonal

• Apply the same transform to :

Note that is symmetric as well as :

• Diagonalize

As is symmetric,, it can be diagonalized by its orthogonal eigenvector matrix :

i.e.,

where we have defined

which is not orthogonal:

• This also diagonalizes :

• Now we have

Right multiplying both sides of the second equation by and equating the left-hand side to that of the first equation, we get

i.e., and are the eigenvalue and eigenvector matrices of the generalized eigenvalue problem. Note, however, as shown above, is not orthogonal.

The Rayleigh quotient of two symmetric matrices and is a function of a vector defined as:

To find the optimal corresponding to the extremum (maximum or minimum) of , we find its derivative with respect to :

Setting it to zero we get

The second equation can be recognized as a generalized eigenvalue problem with being the eigenvalue and and the corresponding eigenvector. Solving this we get the vector corresponding to the maximum/minimum eigenvalue , which maximizes/minimizes the Rayleigh quotient.

Next: Normal matrices and diagonalizability Up: algebra Previous: Eigenvalues and matrix diagonalization
Ruye Wang 2015-04-27