Next: Geometric Interpretation of KLT Up: klt Previous: KLT Completely Decorrelates the

# KLT Optimally Compacts Signal Energy

Now we show that KLT redistributes the energy contained in the signal so that it is maximally compacted into a small number of components after the KLT transform. Let be an arbitrary orthogonal matrix satisfying , and we represent in terms of its column vectors as

Based on this matrix , an orthogonal transform of a given signal vector can be defined as

The inverse transform is:

Now we consider the variances of the signal components before and after the KLT transform:

where can be considered as the dynamic energy or information contained in the ith component of the signal, and the trace of the covariance matrix represents the total energy or information contained in the signal:

Due to the commutativity of trace: , we have:

reflecting the fact that the total energy or information of the signal is conserved after the KLT transform, or any unitary (orthogonal) transform (Parseval's theorem. However, the energy distribution among all components can be very different before and after the KLT transform, as shown below.

We define the energy contained in the first components after the transform as

is a function of the transform matrix . Since the total energy is conserved, is also proportional to the percentage of energy contained in the first components. In the following we will show that is maximized if and only if the transform matrix is the same as that of the KLT:

In other words, the KLT optimally compacts energy into a few components of the signal. Consider

Now we need to find a transform matrix , so that

The constraint is to guarantee that the column vectors in are normalized. This constrained optimization problem can be solved using Lagrange multiplier method by letting the following partial derivative be zero:

(* The last equal sign is due to explanation in the review of linear algebra.) We see that the column vectors of must be the eigenvectors of :

i.e., the transform matrix must be

where 's are the orthogonal eigenvectors of corresponding to eigenvalues ():

Thus we have proved that the optimal transform is indeed KLT, and

where the ith eigenvalue of is also the average (expectation) energy contained in the ith component of the signal. If we choose those that correspond to the largest eigenvalues of : , then will be maximized.

Due to KLT's properties of signal decorrelation and energy compaction, it can be used for data compression by reducing the dimensionality of the data set. Specifically, we carry out the following steps:

1. Find the mean vector and the covariance matrix of the signal vectors .
2. Find the eigenvalues , ) of , sorted in descending order, and their corresponding eigenvectors ).
3. Choose a lowered dimensionality so that the percentage of energy contained is no less than a given threshold (e.g., 95%):

4. Construct an transform matrix composed of the largest eigenvectors of :

and carry out KLT based on :

As the dimension of is less than the dimension of , data compression is achieved for storage or transmission. This is a lossy compression with the error representing the percentage of information lost: . But as these 's are the smallest eigenvalues, the error is small (e.g., 5%).
5. Reconstruct by inverse KLT:

Next: Geometric Interpretation of KLT Up: klt Previous: KLT Completely Decorrelates the
Ruye Wang 2013-04-23