Skip to main content
81 votes
Accepted

Does no correlation imply no causality?

does an absence of correlation imply absence of causality? No. Any controlled system is a counterexample. Without causal relationships control is clearly impossible, but successful control means - ...
conjugateprior's user avatar
70 votes
Accepted

Does correlation = 0.2 mean that there is an association "in only 1 in 5 people"?

The quoted passage is indeed incorrect. A correlation coefficient quantifies the degree of association throughout an entire population (or sample, in the case of the sample correlation coefficient). ...
Kodiologist's user avatar
  • 20.4k
67 votes

When to remove insignificant variables?

Let me first ask this: What is the goal of the model? If you are only interested in predicting if a customer will buy, then statistcal hypothesis tests really aren't your main concern. Instead, you ...
Demetri Pananos's user avatar
58 votes
Accepted

Is there any relationship among cosine similarity, pearson correlation, and z-score?

The cosine similarity between two vectors $a$ and $b$ is just the angle between them $$\cos\theta = \frac{a\cdot b}{\lVert{a}\rVert \, \lVert{b}\rVert}$$ In many applications that use cosine ...
GeoMatt22's user avatar
  • 13k
57 votes

Generate a random variable with a defined correlation to an existing variable(s)

I will describe the most general possible solution. Solving the problem in this generality allows us to achieve a remarkably compact software implementation: just two short lines of ...
whuber's user avatar
  • 328k
52 votes
Accepted

Generating correlated binomial random variables

Binomial variables are usually created by summing independent Bernoulli variables. Let's see whether we can start with a pair of correlated Bernoulli variables $(X,Y)$ and do the same thing. Suppose ...
whuber's user avatar
  • 328k
49 votes

Example where $X$ and $Z$ are correlated, $Y$ and $Z$ are correlated, but $X$ and $Y$ are independent

Intuitive example: $Z = X + Y$, where $X$ and $Y$ are any two independent random variables with finite nonzero variance.
fblundun's user avatar
  • 3,989
44 votes

When A and B are positively related variables, can they have opposite effect on their outcome variable C?

The other answers are truly marvelous - they give real life examples. I want to explain why this can happen despite our intuition to the contrary. See this geometrically! Correlation is the cosine of ...
sds's user avatar
  • 2,246
43 votes

Interview question: If correlation doesn't imply causation, how do you detect causation?

There are a few ways around this. You are right that A/B testing is one of these. The economics Nobel this year was awarded for the pioneering of field experiments in the study of policies against ...
Student's user avatar
  • 938
42 votes
Accepted

Why does independence imply zero correlation?

By the definition of the correlation coefficient, if two variables are independent their correlation is zero. So, it couldn't happen to have any correlation by accident! $$\rho_{X,Y}=\frac{\...
OmG's user avatar
  • 1,119
42 votes
Accepted

Can statistical units measured per thousand inhabitants be bigger than 1000?

This is not a rate per one thousand people, this is the absolute number of people, with one unit equating 1,000 people. So if you see something like 3,258.1, it simply means 3,258,100 people. This is ...
J-J-J's user avatar
  • 4,915
39 votes
Accepted

Such thing as a weighted correlation?

Formula for weighted Pearson correlation can be easily found on the web, StackOverflow, and Wikipedia and is implemented in several R packages e.g. psych, or weights and in Python's statsmodels ...
Tim's user avatar
  • 140k
37 votes

Correlation does not imply causation; but what about when one of the variables is time?

I'll provide another answer, since I think the ones currently provided miss an important point of the statement the physicist made. The quoted statement is: "correlation does not imply causation, ...
Duncan's user avatar
  • 1,302
35 votes

How is it possible to obtain a good linear regression model when there is no substantial correlation between the output and the predictors?

A pair of variables may show high partial correlation (the correlation accounting for the impact of other variables) but low - or even zero - marginal correlation (pairwise correlation). Which means ...
Glen_b's user avatar
  • 286k
34 votes
Accepted

Zero correlation of all functions of random variables implying independence

Using indicator functions of measurable sets like$$f(x)=\mathbb I_A(x)\quad g(x)=\mathbb I_B(x)$$leads to$$\text{cov}(f(X),g(Y))=\mathbb P(X\in A,Y\in B)-\mathbb P(X\in A)\mathbb P(Y\in B)$$therefore ...
Xi'an's user avatar
  • 107k
33 votes

Does causation imply correlation?

Things are definitely nuanced here. Causation does not imply correlation nor even statistical dependence, at least not in the simple way we usually think about them, or in the way some answers are ...
Carlos Cinelli's user avatar
33 votes

Does no correlation imply no causality?

No. Mainly because by correlation you most likely mean linear correlation. Two variables can be correlated nonlinearly, and may show no linear correlation. It's easy to construct an example like that, ...
Aksakal's user avatar
  • 61.8k
32 votes

Difference between Random Forest and Extremely Randomized Trees

ExtraTreesClassifier is like a brother of RandomForest but with 2 important differences. We are building multiple decision trees. For building multiple trees, we need multiple datasets. Best practice ...
Ashish Anand's user avatar
31 votes

When A and B are positively related variables, can they have opposite effect on their outcome variable C?

Yes, two co-occuring conditions can have opposite effects. For example: Making outrageous statements (A) is positively related to being entertaining (B). Making outrageous statements (A) has a ...
Matthew Gunn's user avatar
  • 22.6k
31 votes
Accepted

Does mean centering reduce covariance?

If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then $$ \begin{aligned} \operatorname{Cov}(X + a, Y + b) &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \\ &= E[(X + a - E[X] - E[...
Artem Mavrin's user avatar
  • 4,067
30 votes

Why are random walks intercorrelated?

Your independent processes are not correlated! If $X_t$ and $Y_t$ are independent random walks: A correlation coefficient unconditional on time does not exist. (Don't talk about $\operatorname{Corr}(...
Matthew Gunn's user avatar
  • 22.6k
28 votes

Simple examples of uncorrelated but not independent $X$ and $Y$

I think the essence of some of the simple counterexamples can be seen by starting with a continuous random variable $X$ centered on zero, i.e. $E[X]=0$. Suppose the pdf of $X$ is even and defined on ...
28 votes

When A and B are positively related variables, can they have opposite effect on their outcome variable C?

I've heard this car analogy which applies well to the question: Driving uphill (A) is positively related to the driver stepping on the gas (B) Driving uphill (A) has a negative effect on vehicle ...
congusbongus's user avatar
28 votes

What is the intuitive meaning of having a linear relationship between the logs of two variables?

You just need to take exponential of both sides of the equation and you will get a potential relation, that may make sense for some data. $$\log(Y) = a\log(X) + b$$ $$\exp(\log(Y)) = \exp(a \log(X) +...
Pere's user avatar
  • 6,643
27 votes

What is an example of perfect multicollinearity?

Here are a couple of fairly common scenarios producing perfect multicollinearity, i.e. situations in which the columns of the design matrix are linearly dependent. Recall from linear algebra that this ...
Silverfish's user avatar
  • 23.8k
26 votes
Accepted

What is the best programmatic way for determining whether two variables are linearly or non-linearly or not even related

It is very difficult to achieve what you want programmatically because there are so many different forms of nonlinear associations. Even looking at correlation or regression coefficients will not ...
Robert Long's user avatar
  • 63.9k
25 votes

Analogy of Pearson correlation for 3 variables

It is indeed something. To find out, we need to examine what we know about correlation itself. The correlation matrix of a vector-valued random variable $\mathbf{X}=(X_1,X_2,\ldots,X_p)$ is the ...
whuber's user avatar
  • 328k
25 votes
Accepted

How to test whether a correlation is equal to 1?

I would argue that there is not any testing to do. If the sample correlation is not 1, then you reject $H_0: \rho=1$ with certainty. Having a correlation of 1 means that the points cannot deviate ...
Dave's user avatar
  • 64.7k
25 votes

Why is correlation only defined between two variables?

Pearson correlation is defined as a measure of the linear relationship between two variables. For other relationships, like multidimensional relationships, we use other names. For instance: one could ...
Sextus Empiricus's user avatar
24 votes
Accepted

Correlation between sine and cosine

Since $$\begin{align} \operatorname{Cov}(Y, Z) &= E[(Y - E[Y])(Z - E[Z])] \\ &= E[(Y - {\textstyle \int}_0^{2\pi} \sin x \;dx)(Z - {\textstyle \int}_0^{2\pi} \cos x \;dx)] \\ &= E[(Y - 0)(...
Kodiologist's user avatar
  • 20.4k

Only top scored, non community-wiki answers of a minimum length are eligible