183
votes
Accepted
What happens if the explanatory and response variables are sorted independently before regression?
I'm not sure what your boss thinks "more predictive" means. Many people incorrectly believe that lower $p$-values mean a better / more predictive model. That is not necessarily true (this being a ...
135
votes
What happens if the explanatory and response variables are sorted independently before regression?
If you want to convince your boss, you can show what is happening with simulated, random, independent $x,y$ data. With R:
...
- 9,766
109
votes
What happens if the explanatory and response variables are sorted independently before regression?
Your intuition is correct: the independently sorted data have no reliable meaning because the inputs and outputs are being randomly mapped to one another rather than what the observed relationship was....
- 1,957
81
votes
Accepted
Does no correlation imply no causality?
does an absence of correlation imply absence of causality?
No. Any controlled system is a counterexample.
Without causal relationships control is clearly impossible, but successful control means - ...
- 20.4k
69
votes
Accepted
Does correlation = 0.2 mean that there is an association "in only 1 in 5 people"?
The quoted passage is indeed incorrect. A correlation coefficient quantifies the degree of association throughout an entire population (or sample, in the case of the sample correlation coefficient). ...
- 19.7k
63
votes
When to remove insignificant variables?
Let me first ask this: What is the goal of the model? If you are only interested in predicting if a customer will buy, then statistcal hypothesis tests really aren't your main concern. Instead, you ...
- 33.4k
62
votes
Accepted
Why zero correlation does not necessarily imply independence
Correlation measures linear association between two given variables and it has no obligation to detect any other form of association else.
So those two variables might be associated in several other ...
- 1,533
56
votes
Generate a random variable with a defined correlation to an existing variable(s)
I will describe the most general possible solution. Solving the problem in this generality allows us to achieve a remarkably compact software implementation: just two short lines of ...
- 316k
55
votes
Accepted
Is there any relationship among cosine similarity, pearson correlation, and z-score?
The cosine similarity between two vectors $a$ and $b$ is just the angle between them
$$\cos\theta = \frac{a\cdot b}{\lVert{a}\rVert \, \lVert{b}\rVert}$$
In many applications that use cosine ...
- 12.6k
52
votes
What happens if the explanatory and response variables are sorted independently before regression?
Actually, let's make this really obvious and simple. Suppose I conduct an experiment in which I measure out 1 liter of water in a standardized container, and I look at the amount of water remaining ...
- 5,386
50
votes
Accepted
Generating correlated binomial random variables
Binomial variables are usually created by summing independent Bernoulli variables. Let's see whether we can start with a pair of correlated Bernoulli variables $(X,Y)$ and do the same thing.
Suppose ...
- 316k
49
votes
Example where $X$ and $Z$ are correlated, $Y$ and $Z$ are correlated, but $X$ and $Y$ are independent
Intuitive example: $Z = X + Y$, where $X$ and $Y$ are any two independent random variables with finite nonzero variance.
- 3,929
44
votes
Accepted
Difference between Random Forest and Extremely Randomized Trees
The Extra-(Randomized)-Trees (ET) article contains a bias-variance analysis.
In Fig. 6 (on page 16), you can see a comparison with multiple methods including RF
on six tests (tree classification and ...
- 2,084
44
votes
When A and B are positively related variables, can they have opposite effect on their outcome variable C?
The other answers are truly marvelous - they give real life examples.
I want to explain why this can happen despite our intuition to the contrary.
See this geometrically!
Correlation is the cosine of ...
- 2,226
43
votes
Interview question: If correlation doesn't imply causation, how do you detect causation?
There are a few ways around this. You are right that A/B testing is one of these. The economics Nobel this year was awarded for the pioneering of field experiments in the study of policies against ...
- 938
42
votes
Accepted
Why does independence imply zero correlation?
By the definition of the correlation coefficient, if two variables are independent their correlation is zero. So, it couldn't happen to have any correlation by accident!
$$\rho_{X,Y}=\frac{\...
- 1,077
42
votes
Accepted
Can statistical units measured per thousand inhabitants be bigger than 1000?
This is not a rate per one thousand people, this is the absolute number of people, with one unit equating 1,000 people. So if you see something like 3,258.1, it simply means 3,258,100 people.
This is ...
- 3,123
38
votes
Accepted
Such thing as a weighted correlation?
Formula for weighted Pearson correlation can be easily found on the web, StackOverflow, and Wikipedia and is implemented in several R packages e.g. psych, or weights and in Python's statsmodels ...
Tim♦
- 135k
37
votes
Correlation does not imply causation; but what about when one of the variables is time?
I'll provide another answer, since I think the ones currently provided miss an important point of the statement the physicist made. The quoted statement is:
"correlation does not imply causation, ...
- 1,302
35
votes
How is it possible to obtain a good linear regression model when there is no substantial correlation between the output and the predictors?
A pair of variables may show high partial correlation (the correlation accounting for the impact of other variables) but low - or even zero - marginal correlation (pairwise correlation).
Which means ...
- 277k
34
votes
Accepted
Zero correlation of all functions of random variables implying independence
Using indicator functions of measurable sets like$$f(x)=\mathbb I_A(x)\quad g(x)=\mathbb I_B(x)$$leads to$$\text{cov}(f(X),g(Y))=\mathbb P(X\in A,Y\in B)-\mathbb P(X\in A)\mathbb P(Y\in B)$$therefore ...
- 103k
33
votes
Does no correlation imply no causality?
No. Mainly because by correlation you most likely mean linear correlation. Two variables can be correlated nonlinearly, and may show no linear correlation. It's easy to construct an example like that, ...
- 60k
32
votes
PCA on correlation or covariance?
A common answer is to suggest that covariance is used when variables are on the same scale, and correlation when their scales are different. However, this is only true when scale of the variables isn'...
- 60k
32
votes
Does causation imply correlation?
Things are definitely nuanced here. Causation does not imply correlation nor even statistical dependence, at least not in the simple way we usually think about them, or in the way some answers are ...
- 12k
32
votes
When A and B are positively related variables, can they have opposite effect on their outcome variable C?
Yes, two co-occuring conditions can have opposite effects.
For example:
Making outrageous statements (A) is positively related to being entertaining (B).
Making outrageous statements (A) has a ...
- 22k
31
votes
Accepted
Does mean centering reduce covariance?
If $X$ and $Y$ are random variables and $a$ and $b$ are constants, then
$$
\begin{aligned}
\operatorname{Cov}(X + a, Y + b)
&= E[(X + a - E[X + a])(Y + b - E[Y + b])] \\
&= E[(X + a - E[X] - E[...
- 3,927
29
votes
Why zero correlation does not necessarily imply independence
There is a generalized lack of rigor in the use of the word "correlation" for the simple reason that it can have widely differing assumptions and meanings. The simplest, loosest and most common usage ...
- 10.2k
29
votes
Why are random walks intercorrelated?
Your independent processes are not correlated! If $X_t$ and $Y_t$ are independent random walks:
A correlation coefficient unconditional on time does not exist. (Don't talk about $\operatorname{Corr}(...
- 22k
28
votes
Simple examples of uncorrelated but not independent $X$ and $Y$
I think the essence of some of the simple counterexamples can be seen by starting with a continuous random variable $X$ centered on zero, i.e. $E[X]=0$. Suppose the pdf of $X$ is even and defined on ...
28
votes
Difference between Random Forest and Extremely Randomized Trees
ExtraTreesClassifier is like a brother of RandomForest but with 2 important differences.
We are building multiple decision trees. For building multiple trees, we need multiple datasets. Best practice ...
- 381
Only top scored, non community-wiki answers of a minimum length are eligible