5
$\begingroup$

I've started with thinking that if X and Y are perfectly correlated, then it is the same as looking at the correlation of Y and -Y (since the X provides no new information) and thus the correlation is -1.

Is this correct?

$\endgroup$
  • $\begingroup$ No reason why it can't be 1. Perfect correlation only says that Y=aX+b. If a is positive the correlation is 1 and it is -1 if a is negative. So what you need to do is find the correlation between (1+a)X+b with (1-a)X-b. $\endgroup$ – Michael Chernick Nov 19 '17 at 21:56
  • 2
    $\begingroup$ Your reasoning is flawed. $\endgroup$ – Glen_b Nov 19 '17 at 22:30
  • $\begingroup$ If a=1 the correlation is 0. But if a is not 1 it can be determined using my argument.. $\endgroup$ – Michael Chernick Nov 19 '17 at 23:27
  • 2
    $\begingroup$ If $X$ and $Y$ have a correlation coefficient of $\rho$ then $X$ and $Y$ have a correlation coefficient of $-\rho$. If $X$ and $Y$ are "perfectly correlated" (i.e. knowing $X=x$ you can be almost sure $Y=a+bx$ for some fixed $a,b$) then $\rho=+1$ or $-1$ but without more information you cannot tell which $\endgroup$ – Henry Nov 20 '17 at 8:47
5
$\begingroup$

Hint: In general, \begin{align} \rho_{A,B} &= \frac{\operatorname{cov}(A,B)}{\sqrt{\operatorname{var}(A)\operatorname{var}(B)}},\\ \operatorname{var}(X\pm Y)&= \operatorname{var}(X)+\operatorname{var}(Y) \pm 2\operatorname{cov}(X,Y),\\ \text{and}\qquad\operatorname{cov}(X+Y,X-Y)&=\operatorname{var}(X)-\operatorname{var}(Y)\end{align} So, work out what $\rho_{X+Y,X-Y}$ is in general, and in the special case when $Y = aX+b$. You might be surprised at the result.

$\endgroup$
  • $\begingroup$ Using those properties, I came out to Var(X) - Var(Y) / 0? The denominator all cancelled out but that cannot be right as the denominator being 0 means that the correlation is undefined? $\endgroup$ – JPMSpoof Nov 19 '17 at 23:48
  • 1
    $\begingroup$ @JPMSpoof : You would only get a denominator of zero from assuming equal variance. Perfectly correlated is not the same as equal variance. For instance, $Y = 5X$ yields perfect correlation with unequal variance. $\endgroup$ – Eric Towers Nov 19 '17 at 23:55
  • $\begingroup$ No, the denominator is $\sigma_{X+Y}\sigma_{X-Y}$ which in the special case $\rho_{X,Y}=1$ makes $\sigma_{X\pm Y}= \sigma_{X}\pm \sigma_Y$ leading to $$\rho_{X+Y,X-Y}=\frac{\sigma_X^2-\sigma_Y^2}{(\sigma_X+\sigma_Y)(\sigma_X-\sigma_Y)}=1.$$ $\endgroup$ – Dilip Sarwate Nov 20 '17 at 0:02
  • $\begingroup$ @EricTowers The answer I provided might give some insight into the denominator of zero issue, perhaps? $\endgroup$ – Mark White Dec 2 '17 at 6:27
2
$\begingroup$

I'll treat this as , and I'd encourage you to read its wiki and add the tag.

Your argument is already very good. Here are a few pointers. Feel free to write a comment so we can discuss and work towards a good answer.

  • I assume you are looking at Pearson's correlation, right? (Does your argument work for other measures of correlation?)
  • What does a perfect correlation mean graphically?
  • What will $X+Y$ and $X-Y$ look like graphically if $X$ and $Y$ are perfectly Pearson-correlated?
$\endgroup$
  • $\begingroup$ Yes I am looking at Pearson's correlation. I know perfect correlation means the X-Y scatterplot follows a perfectly straight line, so now I imagine as X+Y increase, and X-Y decrease, the correlation will be negative $\endgroup$ – JPMSpoof Nov 19 '17 at 20:59
  • 2
    $\begingroup$ @Stephan I don't think the argument in the question is good at all. Consider X,Y bivariate normal with zero mean and Var(X) = $10^4$ Var(Y). Then X+Y is approximately X and X-Y is approximately X, and the OP's answer of -1 is as wrong as could be. Now consider zero means and Var(X) = Var(Y), in which case X-Y is always 0 and the correlation is undefined. $\endgroup$ – Glen_b Nov 19 '17 at 22:35
  • $\begingroup$ @Glen_b Please check your assertion that $X-Y$ is always $0$ when $X$ and $Y$ are perfectly correlated (cf. my comment to the OP in response to his comment on my answer). $\endgroup$ – Dilip Sarwate Nov 20 '17 at 0:05
1
$\begingroup$

I see this already has an accepted answer, but I've always liked simulations more than equations, and this seemed like a fun question. I generated a variable $x$ from a distribution $N(0, 1)$ whose sample size $n$ was drawn from $U(100, 10000)$. To make $y$, I simply added a constant—drawn from $U(1, 100)$—to $x$. I then calculated the correlation between $X + Y$ and $X - Y$. I did this 10,000 times:

set.seed(1839)
cors <- sapply(1:10000, function(placeholder) {
  n <- runif(1, 100, 10000)
  b0 <- runif(1, 1, 100)
  x <- rnorm(n)
  y <- b0 + x
  cor(x + y, x - y)
})

You'll get a ton of warnings. I run warnings()[1:5] to get the first five:

Warning messages:
1: In cor(x + y, x - y) : the standard deviation is zero
2: In cor(x + y, x - y) : the standard deviation is zero
3: In cor(x + y, x - y) : the standard deviation is zero
4: In cor(x + y, x - y) : the standard deviation is zero
5: In cor(x + y, x - y) : the standard deviation is zero

We can still look at the histogram of correlations that were defined, calling hist(cors[!is.na(cors)]):

enter image description here

We can also look at how many of those simulations had the $Var(X - Y) = 0$:

set.seed(1839)
sd_is_zero <- sapply(1:10000, function(placeholder) {
  n <- runif(1, 100, 10000)
  b0 <- runif(1, 1, 100)
  x <- rnorm(n)
  y <- b0 + x
  ifelse(var(x - y) == 0, TRUE, FALSE)
})

Then we can call prop.table(table(var_is_zero)) to see how many simulations generated $Var(X - Y) = 0$:

var_is_zero
 FALSE   TRUE 
0.1698 0.8302 

But why were some defined and some undefined? Was it related to the sample size or was it related to the constant?

set.seed(1839)
dat <- as.data.frame(matrix(nrow = 10000, ncol = 3))
colnames(dat) <- c("var_is_zero", "n", "b0")
for (i in 1:10000) {
  n <- runif(1, 100, 10000)
  b0 <- runif(1, 1, 100)
  x <- rnorm(n)
  y <- b0 + x
  dat$var_is_zero[i] = ifelse(var(x - y) == 0, TRUE, FALSE)
  dat$n[i] = n
  dat$b0[i] = b0
}

We can now predict whether or not the variance was zero from the sample size, the constant, and the interaction, looking at the result with summary(glm(var_is_zero ~ n * b0, data = dat, family = binomial())):

Call:
glm(formula = var_is_zero ~ n * b0, family = binomial(), data = dat)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-2.5154   0.1628   0.3039   0.5997   1.2894  

Coefficients:
              Estimate Std. Error z value Pr(>|z|)    
(Intercept) -2.613e-01  1.038e-01  -2.518   0.0118 *  
n           -2.114e-05  1.785e-05  -1.184   0.2362    
b0           5.323e-02  2.973e-03  17.907   <2e-16 ***
n:b0        -4.011e-07  4.968e-07  -0.807   0.4195    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 9111.4  on 9999  degrees of freedom
Residual deviance: 7148.0  on 9996  degrees of freedom
AIC: 7156

Number of Fisher Scoring iterations: 6

It looks like the larger the intercept, the more likely the correlation is to be undefined.

$\endgroup$
  • 2
    $\begingroup$ By y <- b0 + x, you implicitly set $\sigma_Y=\sigma_X$, which is not implied by the condition. The result will be well-defined if you use $\sigma_X\neq\sigma_Y$. $\endgroup$ – user3813057 Dec 27 '17 at 0:49
1
$\begingroup$

If $X$ is a linear function of $Y$ (definition of perfect correlation), then both $X-Y$ and $X+Y$ will be linear functions of $Y$, and therefore are linear functions of each other.

So, $X-Y$ and $X+Y$ are perfectly correlated.

$\endgroup$
  • 1
    $\begingroup$ Obviously if one or both of $X-Y$, $X+Y$ are constant, the answer is undefined. I assumed that's not the case while writing this answer. $\endgroup$ – use_norm_approx Dec 27 '17 at 2:15
  • 2
    $\begingroup$ I still on't see why it can't be -1. $\endgroup$ – Michael Chernick Dec 27 '17 at 3:56
  • $\begingroup$ @MichaelChernick, I meant to say they were perfectly correlated. Thanks. $\endgroup$ – use_norm_approx Dec 28 '17 at 16:55
1
$\begingroup$

$X,Y$ perfectly correlate $\implies Cov(X,Y)=\sqrt{Var(X)Var(Y)}$

$Cov(X+Y,X-Y)=Cov(X,X)-Cov(X,Y)+Cov(Y,X)-Cov(Y,Y)=Var(X)-Var(Y)$

$Var(X+Y)Var(X-Y)=[Var(X)+Var(Y)+2Cov(X,Y)][Var(X)+Var(Y)-2Cov(X,Y)]=[Var(X)+Var(Y)]^2-4Cov(X,Y)Cov(X,Y)=_{\rho_{X,Y}=1}=[Var(X)+Var(Y)]^2-4Var(X)Var(Y)=[Var(X)-Var(Y)]^2$

Hence, $\rho_{X+Y,X-Y}=Cov(X+Y,X-Y)/\sqrt{Var(X+Y)Var(X-Y)}=\pm1$.

As @whuber pointed out, when $Var(X)=Var(Y)$, $\rho_{X+Y,X-Y}$ is undefined.

@Dilip Sarwate has provided the answer in the comment to his post. I added some details.

Actually, if $Var(X)=Var(Y)$, given they are perfectly correlated, $Y=\pm X+b$, hence either $X+Y\equiv0$ or $X-Y\equiv0$. Therefore, $\rho=0$.

$\endgroup$
  • $\begingroup$ When $\operatorname{Var}(X)=\operatorname{Var}(Y)$, the denominator is $0$, whence the correlation is undefined in that case. $\endgroup$ – whuber Dec 27 '17 at 1:44
  • 1
    $\begingroup$ @whuber You are right, I forgot to mention this special case. $\endgroup$ – user3813057 Dec 27 '17 at 20:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.