Multivariate Random Signals

Before reading on, it is highly recommended that you review the basics of multivariate probability theory

A real time signal $x(t)$ can be considered as a random process and its samples $x_m\;\;(m=1, \cdots, N)$ a random vector is the expectation of ${\bf x}$:

$${\bf x}=[ x_1, \cdots, x_{N} ]^T$$

The mean vector of ${\bf x}$ is

$${\bf m}_x\stackrel{\triangle}{=}E({\bf x})=[E(x_1), \cdots, E(x_{N}) ]^T =[\mu_1,\cdots, \mu_{N} ]^T$$

The covariance matrix of ${\bf x}$ is

$${\bf\Sigma}_x\stackrel{\triangle}{=} E[ ({\bf x}-{\bf m}_x)({... ...^2 & \vdots \\ \cdots & \cdots & \ddots \end{array} \right]$$

where $\sigma_{ij}^2\stackrel{\triangle}{=}E(x_ix_j)-\mu_i\mu_j$ is the covariance of two random variables $x_i$ and $x_j$. When $i=j$, $\sigma_{ij}^2$ becomes the variance of $x_i$, $\sigma_i^2\stackrel{\triangle}{=}E(x_i-\mu_i)^2=E(x_i^2)-\mu_i^2$. In general, is the data set is complex, the covariance matrix is Hermitian, i.e., ${\bf\Sigma}_x^{*T}={\bf\Sigma}_x$. In particular if the data set is real, then ${\bf\Sigma}_x^*={\bf\Sigma}_x$ is real and symmetric ${\bf\Sigma}_x^T={\bf\Sigma}_x$.

The correlation matrix of $X$ is defined as

$${\bf R}_x\stackrel{\triangle}{=}E({\bf xx}^T) =\left[ \beg... ...j} & \vdots \\ \cdots & \cdots & \ddots \end{array} \right]$$

where $r_{ij}=E(x_ix_j)=\sigma_{ij}^2+\mu_i\mu_j$. Note that as $\sigma_{ij}=\sigma_{ji}$ and $r_{ij}=r_{ji}$, both ${\bf\Sigma}_X={\bf\Sigma}^T_X$ and ${\bf R}_x={\bf R}^T_x$ are symmetric matrices (Hermitian if ${\bf x}$ is complex).

A signal vector ${\bf x}$ can always be easily converted into a zero-mean vector ${\bf x}'={\bf x}-{\bf m}_x$ with all of its dynamic energy (representing the information contained) conserved. Without loss of generality for convenience, sometimes we can assume ${\bf m}_x=0$ so that ${\bf\Sigma}_x={\bf R}_x$.

After a certain orthogonal transform of a given random vector ${\bf x}$, the resulting vector ${\bf y}={\bf A}^T{\bf x}$ is still random with the following mean and covariance:

$${\bf m}_y = E({\bf y})=E({\bf A}^T {\bf x})={\bf A}^T E({\bf x}) ={\bf A}^T {\bf m}_x$$

$${\bf\Sigma}_y = E({\bf yy}^{T})-{\bf m}_y {\bf m}_y^T =E[({\bf A}^{T}{\bf x})({\bf A}^{T}{\bf x})^{T}] -({\bf A}^T {\bf m}_x) ({\bf A}^T {\bf m}_x)^T$$

$$= E[{\bf A}^{T}({\bf xx}^{T}){\bf A}]-{\bf A}^T {\bf m}_x {\bf m}_x... ...({\bf xx}^{T})-{\bf m}_x {\bf m}_x^T ] {\bf A} ={\bf A}^{T}{\bf\Sigma}_x{\bf A}$$