It might seem obvious that the covariance is a "deeper" property of the data generation process (DGP), since normally the specification of a joint distribution is done in terms of its mean vector and covariance matrix. From these, the correlation structure can be deduced. In fact a correlation is mathematically defined as a function of the mean and covariance.

However, it is also possible to define a multivariate DGP in terms of means and correlations. In fact, correlations are interesting enough in that they are bounded between -1 and 1 and are comparable. Thus, it is possible to see them somewhat as a more "structural" property of the data.

Is covariance, ontologically speaking (if this is possible in statistics), a more structural parameter than correlation? Or are maybe both equally structural?

  • 3
    It's hard for me to see covariance as structural as it depends on the scale. You could transform one of the two variables by measuring it in millimeters vs meters: that would multiply its $SD$ by $1,000$, and would have the same effect on covariance. On the contrary, by changing the scale (in general adding a constant and/or multiplying for a constant) doesn't change correlation that, as you say, is bounded between $-1$ and $+1$, with such extreme values having the meaning of perfect linear dependence. Thus, I'd say the correlation is definitely more informative than the covariance. – Federico Tedeschi May 31 at 10:32

Neither are particularly deep. To really dig into the structure of the relationship between two random variables, you need to examine the full copula. In essence, a copula is what remains of the joint distribution between two random variables once their marginal probabilities have been flattened out to uniform. That such a factorization is always possible is due to Sklar's Theorem.

Here is a scatter plot of a real-world data set that has an approximately Gaussian copula. Gaussian copulas represent a fairly simple relationship structure that can be understood purely i terms of the correlation. (To speak to your specific question, yes, correlation is sometimes a good way to understand the relationship between two random variables - when the copula is approximately Gaussian.)

Fairly Typical Copula

However, many other relationships are possible. From the same real-world project, we also structures like this:

sticky

or this:

bowed

That last one is getting a pretty similar $r^2$ as the first, yet their structure is completely different. Perhaps It's an asymmetrical Gumbel copula, or something other asymmetric copula.

There are infinitely many possible copulas, but these are studied in terms of a finite number of parameterized families. At least two (very important) copulas can be parameterized in terms of their correlation matrices: the Gaussian and the Student's $t$ copulas. But other copulas have other parameters and therefore cannot be understood in terms of correlation or covariance.

So, returning to your question, if you believe that the joint probability distribution between the $k$ random variables in question is well-approximated by a Gaussian or Student's $t$ copula, then yes, correlation is a powerful and sufficient way to understand their structure. But if the copula is from some other family, or unknown, or we're trying to make very general statements about all possible joint probability distributions, then correlation is woefully inadequate.

  • Thanks! A lot to digest. Did not know about copulas. Are you saying that, depending on the joint distribution of the DPG, it might not even be enough to characterise the dependence using cov/corr? That only Gaussian and Student's t allow that possibility? – luchonacho Jul 2 at 8:43
  • Yes, that's right. Most copulas families have only one or two parameters even in high dimensions. (Note that if you have $k$ dimensions, the correlation matrix will have $k(k-1)/2$ degrees of freedom, so Gaussian copulas in general have many parameters.) The parameters of most copula families are not correlations.This is clear from the short list of families on Wikipedia. So if two random variables have some non-elliptical copula, then calculating their $r$ correlation is not very insightful. – olooney Jul 2 at 16:29

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.