Let $\boldsymbol{X}\in\mathbb{R}^n$ be a random vector. Then

$$\text{Cov}(\boldsymbol{X}) = E\left[(\boldsymbol{X}-E[\boldsymbol{X}])^T(\boldsymbol{X}-E[\boldsymbol{X}])\right]=E[\boldsymbol{X}\boldsymbol{X}^T]-\boldsymbol{\mu_X}\boldsymbol{\mu_X}^T$$

Let $\boldsymbol{D} = \text{diag}\left(\text{Cov}(\boldsymbol{X})\right)$. Then


Now, suppose $\boldsymbol{X,Y}\in\mathbb{R}^n$ are random vectors. The cross-covariance is $$\text{Cov}(\boldsymbol{X},\boldsymbol{Y}) = E\left[(\boldsymbol{X}-E[\boldsymbol{X}])^T(\boldsymbol{Y}-E[\boldsymbol{Y}])\right]=E[\boldsymbol{X}\boldsymbol{Y}^T]-\boldsymbol{\mu_X}\boldsymbol{\mu_Y}^T$$

I found all the above facts in this link:


So then, I assumed there was a correlation version of the cross-covariance. But when I looked it up, Wikiepdia says

enter image description here

Source: https://en.wikipedia.org/wiki/Cross-correlation

I don't understand how this follows. Intuitively, this doesn't make sense to me because this Wikipedia formula doesn't involve rescaling. The prior definition for one random vector involves using $\boldsymbol{D}^{-\frac{1}{2}}$ and this makes sense to me because it corresponds to dividing by $\sigma_X$ in the univariate case. This definition on Wikipedia doesn't involve such a standardization, so I don't see how it could be correct.

What is the correct definition of cross-correlation between two random vectors?

  • $\begingroup$ Please read the full article you cite. For more detail it references en.wikipedia.org/wiki/…, which explains the difference between "cross correlation" and "cross covariance," etc. $\endgroup$
    – whuber
    Apr 15 at 22:00
  • $\begingroup$ Even so, there is no intuition offered for how to understand that derivation. Do you have a more comprehensive introductory source perhaps than just that? I tried googling cross correlation, but I keep getting signal processing literature of the form $(f\star g)(\tau)$ and I don't see how that's related to the correlation between two random vectors. $\endgroup$ Apr 16 at 16:47
  • $\begingroup$ There's no derivation: these are two different things. $\endgroup$
    – whuber
    Apr 16 at 17:11
  • $\begingroup$ @whuber ah ok, that makes sense. My confusion is arising from a course I'm taking on canonical correlation analysis. My professor started the discussion by asking us to consider the correlation between two random vectors $X,Y$ and I assumed, just as there is a cross covariance which considers the covariance between two vectors, then cross correlation must be analogous. But I am apparently wrong. Thanks for clarifying. $\endgroup$ Apr 16 at 18:25
  • 1
    $\begingroup$ It's confusing terminology: I had to study several Wikipedia articles carefully. $\endgroup$
    – whuber
    Apr 16 at 19:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.