# Is it possible that X and Y are uncorrelated but X can significantly predict Y? [duplicate]

I was just wondering is it possible that X and Y are uncorrelated but X can significantly predict Y?

If so, how would you explain and interpret that?

• Welcome to Cross Validated! What exactly do you mean? For instance, do you mean if $X$ and $Y$ can be uncorrelated yet $X$ can have a significant parameter in an OLS regression?
– Dave
Jul 16, 2022 at 19:14
• Yes. To be more specific, I used the WITH function in Mplus to run the correlation analysis between X and Y first, which shows no significant correlation. However, when I used ON function (Y as the outcome, X as the predictor), the p-value becomes significant. I didn't add other variables to it. Jul 16, 2022 at 19:40
• Let $X$ be symmetrically distributed around $0$ and let $Y=X^2.$ The latter is perfectly predictable from $X$ but the correlation is zero.
– whuber
Jul 16, 2022 at 20:21
• @EmmaN. What you appear to describe seems unlikely -- both a test of Pearson correlation and a simple linear regression should have the exact same t-value and the exact same p-value; can you give some more details about what exactly you did? (I'm not specifically asking for code - though ultimately that may be where the issue is - but a clear explanation of exactly what you attempted, perhaps you could add a few sentences to your question? It may be that you didn't quite do what you thought you did.) .... how many observations were there? Jul 18, 2022 at 0:55
• More details might help to unpick what happened Jul 19, 2022 at 6:55

Yes.

Take for example the unit circle coordinates for (x, y). They are uncorrelated, yet if you know x you know that y can only take on 2 values - or 1 exactly (y=0) if x =1 or -1.

More generally marginal independence does not imply conditional indedepedence. That is there might be a third variable z that allows you to predict y from x knowing z. See Examples of marginal independence, conditional dependence

• This is interesting, but I take the question to ask if $X$ alone can predict $Y$.
– Dave
Jul 16, 2022 at 19:15
• Fair. That's why I included the first (toy) example to give specific (x,y) example without the conditional example. Jul 16, 2022 at 19:17
• Dave: yep, I meant X alone can predict Y. Jul 16, 2022 at 19:41
• Another example is Y = cos(X) over -pi...pi, where X directly determines Y but they are uncorrelated.
– jpa
Jul 17, 2022 at 9:00
• @LorenPechtel As long as the range is symmetric.
– jpa
Jul 18, 2022 at 6:17

Let's prove the fact that @whuber gave in a comment to the question:

Let $$X$$ be symmetrically distributed around $$0$$ and let $$Y=X^2$$. The latter is perfectly predictable from $$X$$ but the correlation is zero.

We first note that $$X$$ obviously determines its square $$X^2=Y$$. The proof of the second part is more interesting:

By definition, a random variable $$X$$ has a symmetric distribution about $$\mu$$ if $$X-\mu$$ has the same distribution as $$\mu-X$$. For symmetry about $$\mu=0$$ we get that $$X-0=X$$ has the same distribution as $$0-X=-X$$.

Using that $$X$$ and $$-X$$ have the same distribution and thus the same moments, the fact that $$(-1)^a=-1$$ for any odd number $$a$$, and linearity of expectation, we get $$\mathbb{E}(X^a)=\mathbb{E}[(-X)^a]=\mathbb{E}(-X^a)=-\mathbb{E}(X^a)$$ and hence $$\mathbb{E}(X^a)=0$$ for any odd number $$a$$.

For the covariance between $$X$$ and $$Y$$ we have $$\mathrm{Cov}(X,Y)=\mathbb{E}(XY)-\mathbb{E}(X)\mathbb{E}(Y)=\mathbb{E}(X^3)-\mathbb{E}(X^1)\mathbb{E}(X^2)$$ and using the previous result $$\mathbb{E}(X^a)=0$$ for $$a\in\{1,3\}$$ it follows that $$\mathrm{Cov}(X,Y)=0$$, which is equivalent to $$\mathrm{Corr}(X,Y)=0$$.

This is not particularly surprising as the covariance and correlation are measures of linear association between two random variables.

• thank you for the answer, but I can't seem to understand it... 😭 Jul 19, 2022 at 2:28
• @EmmaN. can you tell me which part(s) you don't understand? Jul 20, 2022 at 1:51
• the equation. ok, I have to admit that I asked this question because I found something weird in my stats analysis: one result shows that X and Y are unrelated (model 1), but the other result shows that X can predict Y (model 2). However, it turned out it was my misunderstanding or that I should not interpret the results as I thought. Model 1 is an SEM measurement model that includes 5 latent variables (X and Y being two of them). Model 2 is a structural model that also includes the same 5 variables but Y as the outcome variable (thus endogenous). Jul 20, 2022 at 6:02

Correlation measures a certain kind of relation (usually linear) between variables, but other relationships (nonlinear) are possible as well. This is discussed in depth in the Why zero correlation does not necessarily imply independence thread. So it is possible that there’s a non-linear relationship between the two variables, that makes them non-independent, yet it is not measured by correlation.

• Is a linear regression also possible? Jul 16, 2022 at 19:59
• @EmmaN. what do you mean? Linear regression is just scaled correlation en.wikipedia.org/wiki/Simple_linear_regression
– Tim
Jul 16, 2022 at 20:43
• @EmmaN. If correlation is $0$ between two variables, then a simple linear regression will give a horizontal line. Jul 18, 2022 at 15:39

There are a few ways this can happen. Five come to mind. In your case, I suspect the fifth.

1) The relationship between $$X$$ and $$Y$$ is nonlinear

As some other answers and comments have discussed, correlation deals with linear relationships. If the relationship is nonlinear, correlation can miss that there is a relationship.

set.seed(2022)
N <- 100
a <- 1
x <- seq(-a, a, 2*a/(N - 1))
y <- x^2
plot(x, y)
cor(x, y) In this example, $$X$$ perfectly predicts $$Y$$. However, the correlation is zero.

The remedy is to consider the quadratic term in a regression, such as L <- lm(y ~ x + I(x^2)). Such a regression gives the expected result of perfect predictability of $$Y$$.

2) Your WITH and ON functions use different tests

I don't know your software, but there are multiple ways of testing hypotheses. In a theoretical statistics class, such as one taught from the Casella/Berger book Statistical Inference, you will learn about what I call the "big three" hypothesis tests: Wald, likelihood ratio, and score (sometimes called Lagrange multiplier). All of these can give slightly different results. If WITH and ON use different tests of the same null hypothesis, it is possible for one to give a "significant" $$p=0.043$$ and the other to give an "insignificant" $$p = 0.052$$.

3) You've included some other variable in the regression

Consider the following picture. As you might expect, the correlation is zero.

However, I made this by having two groups!

set.seed(2022)
N <- 100
a <- 1
x <- seq(-a, a, 2*a/(N - 1))
g <- rep(c(0, 1), N/2)
y <- x - 2*g*x
plot(x, y)
cor(x, y)


If we consider the group variable g and an interaction between x and g, we get perfect predictability.

L <- lm(y ~ x + g + x*g) If we know we're dealing with a blue group or a red group, we can make perfect predictions and get perfect correlation.

set.seed(2022)
N <- 100
a <- 1
x <- seq(-a, a, 2*a/(N - 1))
g <- rep(c(0, 1), N/2)
y <- x - 2*g*x
plot(x[g==0], y[g==0], col = 'red')
points(x[g==1], y[g==1], col = 'blue')
plot(x[g==0], y[g==0], col = 'red', main = paste("Red correlation is", cor(x[g==0], y[g==0])))
plot(x[g==1], y[g==1], col = 'blue', main = paste("Blue correlation is", cor(x[g==1], y[g==1])))  4) You have a false positive (type I error)

Hypothesis tests falsely reject true null hypothesis some proportion of the time. Just because you know that, at the population level, $$X$$ and $$Y$$ are uncorrelated does not mean that the regression on the sample will reflect that fact.

set.seed(2022)
N <- 250
R <- 1000
ps <- rep(NA, R)
for (i in 1:R){

x <- rnorm(N)
y <- rnorm(N)
L <- lm(y ~ x)
ps[i] <- summary(L)\$coef[2, 4] # p-value on the slope coefficient
}

ecdf(ps)(0.05)


Even though we know that $$X$$ and $$Y$$ are uncorrelated for each iteration in the above simulation, about $$5\%$$ of the tests have p-values of $$0.05$$ or lower, so you have type I errors occur about $$5\%$$ of the time.

5) You exclude an intercept term, which is what I suspect happened to you

I know one of the common Python regression packages (I forget if it is sklearn or statsmodels) excludes an intercept by default, so Mplus might do the same. Look at the following image of uncorrelated $$X$$ and $$Y$$. If we omit the intercept from the regression, then we force the regression line to go through $$(0,0)$$. The regression $$\hat y_i = \hat\beta x_i$$ has a highly significant $$\hat\beta$$.

set.seed(2022)
N <- 10000
x <- rnorm(N, 10, 1)
y <- rnorm(N, 10, 1)
plot(x, y)
plot(x, y, xlim = c(0, 14), ylim = c(0, 14))
points(0, 0, col = 'blue')
plot(x, y, xlim = c(0, 14), ylim = c(0, 14))
abline(a = 0, b = 1, col = 'red')
points(0, 0, col = 'blue')
L <- lm(y ~ 0 + x)
summary(L)
cor.test(x, y)


However, when we include the intercept, the slope, correctly, lacks significance.

L <- lm(y ~ x)
summary(L)
Call:
lm(formula = y ~ x)

Residuals:
Min      1Q  Median      3Q     Max
-3.8865 -0.6757  0.0123  0.6770  3.7909

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 10.093635   0.099646   101.3   <2e-16 ***
x           -0.007935   0.009924    -0.8    0.424
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.9891 on 9998 degrees of freedom
Multiple R-squared:  6.395e-05, Adjusted R-squared:  -3.607e-05
F-statistic: 0.6394 on 1 and 9998 DF,  p-value: 0.424


HOWEVER, you are correct in that the usual thinking is that correlation and simple linear regression are the same. That is, if $$X$$ and $$Y$$ have a correlation that you test somehow (Wald is the usual) and get some p-value $$p$$, then when you regress $$Y$$ on $$X$$ (with an intercept) and test the slope with the same test (Wald is the usual), you should get the same p-value.

• A big +1. The graphics tell the story well.
– whuber
Jul 19, 2022 at 20:23
• thanks so much for the visual explanation. I think I might find the answers to my confusion: one result shows that X and Y are unrelated (model 1), but the other result shows that X can predict Y (model 2). However, Model 1 is an SEM measurement model that includes 5 latent variables (X and Y being two of them). Model 2 is a structural model that also includes the same 5 variables but Y as the outcome variable (thus endogenous). So the two different results came out of two different contexts, and should not be interpreted the same way as my question suggests. (I'm terrible at stats...) Jul 20, 2022 at 6:08