Questions tagged [correlation]
A measure of the degree of association among a pair of variables.
869
questions
212
votes
7
answers
204k
views
PCA on correlation or covariance?
What are the main differences between performing principal component analysis (PCA) on the correlation matrix and on the covariance matrix? Do they give the same results?
- 2,300
148
votes
9
answers
320k
views
What is the difference between linear regression on y with x and x with y?
The Pearson correlation coefficient of x and y is the same, whether you compute pearson(x, y) or pearson(y, x). This suggests that doing a linear regression of y given x or x given y should be the ...
- 3,263
148
votes
6
answers
296k
views
Correlations with unordered categorical variables
I have a dataframe with many observations and many variables. Some of them are categorical (unordered) and the others are numerical.
I'm looking for associations between these variables. I've been ...
- 1,797
66
votes
5
answers
258k
views
Correlations between continuous and categorical (nominal) variables
I would like to find the correlation between a continuous (dependent variable) and a categorical (nominal: gender, independent variable) variable. Continuous data is not normally distributed. Before, ...
155
votes
8
answers
294k
views
Pearson's or Spearman's correlation with non-normal data
I get this question frequently enough in my statistics consulting work, that I thought I'd post it here. I have an answer, which is posted below, but I was keen to hear what others have to say.
...
- 45k
46
votes
4
answers
35k
views
Bound for the correlation of three random variables
There are three random variables, $x,y,z$. The three correlations between the three variables are the same. That is,
$$\rho=\textrm{cor}(x,y)=\textrm{cor}(x,z)=\textrm{cor}(y,z)$$
What is the ...
- 561
107
votes
9
answers
88k
views
Generate a random variable with a defined correlation to an existing variable(s)
For a simulation study I have to generate random variables that show a predefined (population) correlation to an existing variable $Y$.
I looked into the R ...
- 4,700
9
votes
1
answer
35k
views
Covariance of a variable and a linear combination of other variables
Let $X,A,B,C,D$ be time-series variables and the covariance between any two pairs of these are known.
Suppose we want to find $\textrm{cov}(X,aA + bB + cC + dD)$, where $a,b,c,d$ are constants.
Is ...
user14281
146
votes
6
answers
85k
views
Should one remove highly correlated variables before doing PCA?
I'm reading a paper where author discards several variables due to high correlation to other variables before doing PCA. The total number of variables is around 20.
Does this give any benefits? It ...
- 1,571
101
votes
1
answer
121k
views
What correlation makes a matrix singular and what are implications of singularity or near-singularity?
I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
- 1,431
162
votes
9
answers
50k
views
Does causation imply correlation?
Correlation does not imply causation, as there could be many explanations for the correlation. But does causation imply correlation? Intuitively, I would think that the presence of causation means ...
- 1,723
78
votes
9
answers
42k
views
If A and B are correlated with C, why are A and B not necessarily correlated?
I know empirically that is the case. I have just developed models that run into this conundrum. I also suspect it is not necessarily a yes/no answer. I mean by that if both A and B are correlated ...
- 7,732
19
votes
2
answers
24k
views
Correlation coefficient between a (non-dichotomous) nominal variable and a numeric (interval) or an ordinal variable
I've already read all the pages in this site trying to find the answer to my problem but no one seems to be the right one form me...
First I explain you the kind of data I'm working with...
Let's ...
- 193
161
votes
5
answers
620k
views
How to choose between Pearson and Spearman correlation?
How do I know when to choose between Spearman's $\rho$ and Pearson's $r$? My variable includes satisfaction and the scores were interpreted using the sum of the scores. However, these scores could ...
user3636
62
votes
4
answers
67k
views
How to generate correlated random numbers (given means, variances and degree of correlation)?
I'm sorry if this seems a bit too basic, but I guess I'm just looking to confirm understanding here. I get the sense I'd have to do this in two steps, and I've started trying to grok correlation ...
- 731
102
votes
6
answers
98k
views
Kendall Tau or Spearman's rho?
In which cases should one prefer the one over the other?
I found someone who claims an advantage for Kendall, for pedagogical reasons, are there other reasons?
- 21.6k
31
votes
7
answers
41k
views
Testing for linear dependence among the columns of a matrix
I have a correlation matrix of security returns whose determinant is zero. (This is a bit surprising since the sample correlation matrix and the corresponding covariance matrix should theoretically be ...
- 3,091
108
votes
1
answer
204k
views
Correlation between a nominal (IV) and a continuous (DV) variable
I have a nominal variable (different topics of conversation, coded as topic0=0 etc) and a number of scale variables (DV) such as the length of a conversation.
How can I derive correlations between ...
- 1,081
40
votes
4
answers
61k
views
X and Y are not correlated, but X is significant predictor of Y in multiple regression. What does it mean?
X and Y are not correlated (-.01); however, when I place X in a multiple regression predicting Y, alongside three (A, B, C) other (related) variables, X and two other variables (A, B) are significant ...
- 5,064
43
votes
2
answers
73k
views
Variance of product of dependent variables
What is the formula for variance of product of dependent variables?
In the case of independent variables the formula is simple:
$$ \operatorname{var}(XY) = E(X^2Y^2) - E(XY)^2 = \operatorname{var}(X) \...
- 133
105
votes
17
answers
72k
views
Under what conditions does correlation imply causation?
We all know the mantra "correlation does not imply causation" which is drummed into all first year statistics students. There are some nice examples here to illustrate the idea.
But sometimes ...
- 56.8k
90
votes
4
answers
181k
views
How to use Pearson correlation correctly with time series
I have 2 time-series (both smooth) that I would like to cross-correlate to see how correlated they are.
I intend to use the Pearson correlation coefficient. Is this appropriate?
My second question ...
- 1,203
21
votes
1
answer
34k
views
What is the proper association measure of a variable with a PCA component (on a biplot / loading plot)?
I am using FactoMineR to reduce my data set of measurements to the latent variables.
The variable map above is clear for me to interpret, but I am confused when ...
- 355
17
votes
3
answers
30k
views
What are the assumptions of factor analysis?
I want to check if I really understood [classic, linear] factor analysis (FA), especially assumptions that are made before (and possibly after) FA.
Some of the data should be initially correlated and ...
- 343
103
votes
9
answers
287k
views
What's the difference between correlation and simple linear regression?
In particular, I am referring to the Pearson product-moment correlation coefficient.
- 9,902
87
votes
7
answers
167k
views
Is it meaningful to calculate Pearson or Spearman correlation between two Boolean vectors?
There are two Boolean vectors, which contain 0 and 1 only. If I calculate the Pearson or Spearman correlation, are they meaningful or reasonable?
- 1,045
114
votes
4
answers
275k
views
How does the correlation coefficient differ from regression slope?
I would have expected the correlation coefficient to be the same as a regression slope (beta), however having just compared the two, they are different. How do they differ - what different information ...
- 14.3k
57
votes
7
answers
63k
views
Why zero correlation does not necessarily imply independence
If two variables have 0 correlation, why are they not necessarily independent? Are zero correlated variables independent under special circumstances ? If possible, I am looking for an intuitive ...
- 6,565
73
votes
4
answers
24k
views
Look and you shall find (a correlation)
I have several hundred measurements. Now, I am considering utilizing some kind of software to correlate every measure with every measure. This means that there are thousands of correlations. Among ...
- 905
22
votes
1
answer
4k
views
How to understand the correlation coefficient formula?
Can anyone help me understand the Pearson correlation formula?
the sample $r$ = the mean of the products of the standard scores of variables $X$ and $Y$.
I kind of understand why they need to ...
- 321
34
votes
4
answers
15k
views
Generating correlated binomial random variables
I was wondering if it might be possible to generate correlated random binomial variables following a linear transformation approach?
Below, I tried something simple in ...
- 3,986
110
votes
2
answers
195k
views
What is covariance in plain language?
What is covariance in plain language and how is it linked to the terms dependence, correlation and variance-covariance structure with respect to repeated-measures designs?
- 1,811
60
votes
6
answers
152k
views
Relationship between $R^2$ and correlation coefficient
Let's say I have two 1-dimensional arrays, $a_1$ and $a_2$. Each contains 100 data points. $a_1$ is the actual data, and $a_2$ is the model prediction. In this case, the $R^2$ value would be:
$$
R^2 = ...
- 1,395
34
votes
1
answer
14k
views
Geometric interpretation of multiple correlation coefficient $R$ and coefficient of determination $R^2$
I am interested in the geometric meaning of the multiple correlation $R$ and coefficient of determination $R^2$ in the regression $y_i = \beta_1 + \beta_2 x_{2,i} + \dots + \beta_k x_{k,i} + \...
- 23.4k
15
votes
3
answers
6k
views
The equivalence of sample correlation and R statistic for simple linear regression
It is often stated that the square of the sample correlation $r^2$ is equivalent to the $R^2$ coefficient of determination for simple linear regression. I have been unable to demonstrate this myself ...
- 193
50
votes
8
answers
89k
views
Simple examples of uncorrelated but not independent $X$ and $Y$
Any hard-working student is a counterexample to "all students are lazy".
What are some simple counterexamples to "if random variables $X$ and $Y$ are uncorrelated then they are independent"?
19
votes
3
answers
14k
views
What to do with random effects correlation that equals 1 or -1?
Not so uncommon occurrence when dealing with complex maximal mixed models (estimating all possible random effects for given data and model) is perfect (+1 or -1) or nearly perfect correlation among ...
- 1,722
14
votes
2
answers
28k
views
Positive correlation and negative regressor coefficient sign
Is it possible to obtain a positive correlation between a regressor and a response (+0,43) and, after that, obtain a negative coefficient in the fitted regression ...
- 231
7
votes
1
answer
7k
views
How to infer correlations from correlations
I have a question regarding correlation inference. Consider, I have two sets of variables X and Y. For an x element of X I know the correlation to an unknown variable z. I also have the covariance ...
- 71
36
votes
3
answers
36k
views
Generating data with a given sample covariance matrix
Given a covariance matrix $\boldsymbol \Sigma_s$, how to generate data such that it would have the sample covariance matrix $\hat{\boldsymbol \Sigma} = \boldsymbol \Sigma_s$?
More generally: we are ...
- 1,674
4
votes
2
answers
4k
views
Can I analyze or model a conditional correlation?
In my research I'm looking at the correlation between self-harm and aggression (both continuous). Now, I also have some variables (e.g. depressive symptoms; also continuous) which I do believe ...
- 41
53
votes
4
answers
23k
views
Why does the correlation coefficient between X and X-Y random variables tend to be 0.7
Taken from Practical Statistics for Medical Research where Douglas Altman writes in page 285:
...for any two quantities X and Y, X will be correlated with X-Y.
Indeed, even if X and Y are ...
- 1,507
28
votes
3
answers
7k
views
If linear regression is related to Pearson's correlation, are there any regression techniques related to Kendall's and Spearman's correlations?
Maybe this question is naive, but:
If linear regression is closely related to Pearson's correlation coefficient, are there any regression techniques closely related to Kendall's and Spearman's ...
- 3,929
26
votes
4
answers
36k
views
Averaging correlation values
Let's say I test how variable Y depends on variable X under different experimental conditions and obtain the following graph:
...
- 2,707
12
votes
1
answer
8k
views
Are estimates of regression coefficients uncorrelated?
Consider a simple regression (normality not assumed): $$Y_i = a + b X_i + e_i,$$ where $e_i$ is with mean $0$ and standard deviation $\sigma$. Are the Least Square Estimates of $a$ and $b$ ...
- 581
44
votes
1
answer
22k
views
Manually calculated $R^2$ doesn't match up with randomForest() $R^2$ for testing new data
I know this is a fairly specific R question, but I may be thinking about proportion variance explained, $R^2$, incorrectly. Here goes.
I'm trying to use the ...
- 4,293
36
votes
2
answers
19k
views
Understanding distance correlation computations
As far as I understood, distance correlation is a robust and universal way to check if there is a relation between two numeric variables. For example, if we have a set of pairs of numbers:
...
- 584
13
votes
2
answers
2k
views
Is there a mathematical proof for change being correlated with baseline value
It is shown in answer here and at other places that difference of 2 random variables will be correlated with baseline. Hence baseline should not be a predictor for change in regression equations. It ...
- 10k
6
votes
3
answers
13k
views
Tool for generating correlated data sets
Does anyone know of a tool that I can use to generate a set of data with known correlations (and to put the icing on the cake - output this in json,csv,txt or some common format)?
I am working on ...
- 61
39
votes
9
answers
9k
views
What is the relationship between $Y$ and $X$ in this plot?
What is the relationship between $Y$ and $X$ in the following plot?
In my view there is negative linear relationship, But because we have a lot of outliers, the relationship is very weak. Am I right?
...
- 843