8
$\begingroup$

I faced a limiting distribution with zero covariance between two variables but their correlation is $1$. Is there such a distribution? How it can be explained?


You are right may I need give more detail. OK, X and Y are bivariate normal distribution with different variances and means (free of n) but corr=1-(1/n), now investigate the limiting distribution of Yn|Xn=x.

$\endgroup$
5
  • 24
    $\begingroup$ That distribution is called a computational error. $\endgroup$ Oct 16, 2016 at 6:56
  • 5
    $\begingroup$ Please give the details in order to resolve the apparent discrepancy. What are the circumstances? $\endgroup$
    – Glen_b
    Oct 16, 2016 at 10:36
  • $\begingroup$ Please provide still more detail about the joint distribution of $X_n$ and $Y_n$. In particular, what is giving rise to $\rho_n=1-1/n$? $\endgroup$
    – Mico
    Oct 17, 2016 at 3:47
  • $\begingroup$ Unfortunately, I do not have more detail. Your question is a question that I was thinking about too. How ρn depends on n when variances are free of n? and what does it mean exactly? $\endgroup$
    – Behgol
    Oct 17, 2016 at 5:08
  • $\begingroup$ Why do you think the covariance is $0$? $\endgroup$ Oct 17, 2016 at 16:15

4 Answers 4

5
$\begingroup$

Following a clarification by the OP, it appears that a) we assume that the two variables follow jointly a bivariate normal and b) our interest is in the conditional distribution, which is then

$$Y_n\mid X_n=x \ \sim\ \mathcal{N}\left(\mu_y+\frac{\sigma_y}{\sigma_x}\rho_n( x - \mu_x),\, (1-\rho_n^2)\sigma_y^2\right)$$

Then we see that as $n \to \infty$, we have $\rho_n \to 1$, and the variance of the conditional distribution goes to zero. Intuitively, if correlation goes to unity, "knowing $x$" is enough to "know $y$" also.

But nowhere in the above do we get that $\text{Cov}(Y_n, X_n)$ is zero. Even at the limit covariance will remain equal to $\text{Cov}(Y_n, X_n) \to \sigma_y \sigma_x$.

Note that the conditional covariance (and then also the conditional correlation) is always zero, because,

$$\text{Cov}(Y_n, X_n \mid X_n =x) = E(Y_nX_n\mid X_n =x) - E(Y\mid X_n =x) E(X\mid X_n =x)$$

$$=xE(Y_n\mid X_n =x) - xE(Y\mid X_n =x) =0$$

This happens because by examining $X_n = x$ we have turned one of the random variables into a constant, and constants do not co-vary with anything.

$\endgroup$
2
20
$\begingroup$

Since the covariance depends on the scale of $X$ and $Y$ and the correlation does not (rescaled to $[-1, -1]$) it is possible. For example, if the variance decreases towards zero:

If $X=Y$ and $\sigma_x^2$ is the variance of $X$, then $\lim_{\sigma_x^2 \to 0} \operatorname{cov}(X, Y) = 0$ and $\lim_{\sigma_x^2 \to 0} \operatorname{cor}(X, Y) = 1$.

Note 1: when $\sigma_x^2 = 0$ the correlation is strictly undefined because its denominator would be equal to 0.

$\endgroup$
2
  • $\begingroup$ You are right may be I should give more detail. OK X and Y arebivariant normal distribution with different variance and mean (free of n) but corr=1-(1/n), now investigate the limiting distribution of Yn|Xn=x. $\endgroup$
    – Behgol
    Oct 16, 2016 at 19:25
  • $\begingroup$ The wording "Since the covariance depends on the scale" implies that this is given in the question. However, that seems to be more than the question implies. It seems to me that you are postulating that this might be so, with stated conclusions. Do correct me if that is wrong. $\endgroup$
    – Nick Cox
    Oct 17, 2016 at 16:02
18
$\begingroup$

As far as I can see (perhaps outside of some special circumstances, but you don't mention any), it's not possible.

The correlation is the covariance divided by the product of the two standard deviations, so if the covariance is zero, the correlation is either zero (when both standard deviations are non-zero) or undefined (when at least one standard deviation is 0). It should not be 1 when the covariance is 0.

I expect you have either made some error in your analysis or your description is insufficiently clear to discern the situation correctly.

$\endgroup$
1
$\begingroup$

You are probably having difficulty because you are visualizing the data as being Gaussian.

It's possible that all data represent the same point (though it would be redundant) and that you have two variables with different names (aliases of each other) comprising the data. This would lead to zero covariance, and a correlation of 1 as fundamentally, covariance represents how spread out the data is across the feature space, while correlation represents how much one variable depends on another, or the degree of influence they have on each other. If the data is not spread out at all, then covariance must be zero.

NOTE However the best thing you can do with such a dataset is simply predict all points as having the same output, which is most likely going to give a high bias

$\endgroup$
2
  • 2
    $\begingroup$ There seem to be quite a few different things going on in this answer, and I have a hard time seeing the relationship. For example, how is paragraph 1 relevant? How is paragraph 3 relevant? Also, how do you arrive at zero covariance in paragraph 2? $\endgroup$ Oct 16, 2016 at 9:50
  • $\begingroup$ Thanks @Richard Hardy for pointing it out. One of the other answers initially suggested a Gaussian solution. That's why para 1. In para 3, I'm just giving my view on what he can do with such a dataset. Fundamentally, covariance represents how spread out the data is across the feature space. If the data is not spread out at , then covariance must be zero. I've added this to the answer too $\endgroup$ Oct 16, 2016 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.