Next: Karhunen-Loeve Transform (KLT) Up: klt Previous: Multivariate Random Signals

# Covariance and Correlation

Let and be two random variables in a random vector . The mean and variance of a variable and the covariance and correlation coefficient (normalized correlation) between two variables and are defined below:

• Mean of :
• Variance of :
• Covariance of and :
• Correlation coefficient between and :
Note that the correlation coefficient can be considered as the normalized version of the covariance .

To obtain these parameters as expectations of the first and second order functions of the random variables, the joint probability density function is required. However, when is not available, the parameters can still be estimated by averaging the outcomes of a random experiment involving these variables repeated times:

To understand intuitively the meaning of these parameters, we consider the following examples under various situations.

Examples:

• Assume the experiment concerning and is repeated times with the following outcomes:

The means, variances and covariance of and can be estimated as

and the correlation coefficient is:

We see that and are highly (maximally in this case) correlated.

• Assume the outcomes of the 3 experiments are

then we have

and

We see that the two variables and can be individually scaled while their correlation remains the same.
• Assume the outcomes of the 3 experiments are

We have and

And the correlation coefficient is:

indicating that the two variables are highly inversely correlated.

• Assume the outcomes are:

We have , and

and , indicating that the two variables are totally uncorrelated (unrelated).

• Assume the outcomes are:

We have and

and , indicating that the two variables are totally uncorrelated (unrelated).

• Assume the experiment is carried out times (combination of examples 4 and 5) with the outcomes:

We still have

and , indicating that the two variables are totally uncorrelated (unrelated).

Now we see that the covariance represents how much the two variables and are positively correlated if , negatively correlated if , or not correlated at all if .

More generally, if a random vector is composed of samples of a random signal , then we can predict that the elements near the main diagonal of the covariance matrix have higher values than those farther away from the diagonal, due to the fact that signal samples close to each other tend to be more correlated than those that are far away.

Next: Karhunen-Loeve Transform (KLT) Up: klt Previous: Multivariate Random Signals
Ruye Wang 2013-04-23