I am trying to get the correlation between 2 variables, given a set of data. Once in a while, in the given data set,When one of the variables has a constant value, since the standard deviation of that variable is zero, I get an NA value for the correlation. (In R). I would like to assign a value for the correlation in these scenarios explicitly or try to get some value through alternate means, so that I am able to compare this point with other times I compute the correlation. How do I go about it? (1) Should I add some noise to that variable and compute the correlation again. Would that be a meaningful thing to do?
$\begingroup$
$\endgroup$
2
-
3$\begingroup$ Note that response constant despite variable predictor and predictor constant while response variable are substantively totally different situations, You shouldn't want to treat them the same way, regardless of the fact that correlation in either case is (fairly reported as) indeterminate. (For completeness, add both variables constant. $\endgroup$– Nick CoxCommented Mar 13, 2017 at 12:09
-
$\begingroup$ Can you expand on what you mean by comparing it with other times you compute the correlation? $\endgroup$– mdeweyCommented Mar 13, 2017 at 12:29
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
Recall that correlation is defined as
$$ \rho_{X,Y}= \frac{\sigma(X,Y)}{\sigma_X \sigma_Y} $$
This means that if one of your "variables" is constant, then it is not a variable, it has variance equal to zero and so, it's correlation with anything is undefined (since you are dividing by zero).
Standard deviation of variable $X$ plus constant $c$ is the same as standard deviation of $X$
$$ \sigma(X + c) = \sigma(X) $$
the same for covariance
$$ \sigma(X + c, Y) = \sigma(X, Y) $$
so adding noise to your constant "variable" would result with measuring correlation of your noise with some other variable (your "variable" is $c$ and noise is $X$).
On another hand, covariance of random variable with constant is zero
$$ \sigma(Y, c) = 0 $$
and constant random variable is independent of any other random variable. So if you really need to re-define correlation for such case then the best choice would be $0$. Notice however that, as noted by Nick Cox in the comment below, this does not solve any of your problems.
The basic problem with constant random variable is that it is independent of everything else and it will not change anything about your analysis. Because of this, many software packages would return errors when using constant variables, or drop them automatically from your analysis. This is what R does and such behavior is consistent with the definition of correlation.
-
6$\begingroup$ Despite the logic here, "correlation undefined" seems the better answer for anyone facing this question. "We don't know" is not the same as zero correlation. Furthermore, further analysis treating the correlation as zero is more likely to mess up any analysis downstream, e.g. PCA based on the correlation matrix with fudged zeros. $\endgroup$– Nick CoxCommented Mar 13, 2017 at 11:55
-
4$\begingroup$ So, if the covariance is 0, then the numerator is 0. And if one variable is constant, then the denominator is 0. And 0/0 is a mess. However, here the denominatior is just a scaling factor, so, maybe corr = 0 is OK. But, while you are surely right that the covariance of a variable with a constant is defined as 0, it's not clear to me that that makes sense, either. substantively. So NA is best, I think $\endgroup$ Commented Mar 13, 2017 at 16:05
-
1$\begingroup$ @PeterFlom I totally agree with you. $\endgroup$– TimCommented Mar 13, 2017 at 16:13
-
1$\begingroup$ This is what R has to say on the matter : ` cor(x <- rep(1, 10), y <- rnorm(10)) [1] NA Warning message: In cor(x <- rep(1, 10), y <- rnorm(10)) : the standard deviation is zero` $\endgroup$ Commented Sep 25, 2020 at 14:53
$\begingroup$
$\endgroup$
Questions about how to do things in R (or any language) are off-topic here, but you also have a statistical question, i.e.
What is a reasonable value for the correlation between two variables when one variable is constant?
You suggest adding some noise to the variable. If you are going to do that, then you might as well simply say that the correlation is 0.
The trouble is that you really don't have any idea what the correlation should be - it could be anything from -1 to 1. That's why R gives NA. So, there is no really reasonable thing to do except to say that "we have no information" and not compare it to other values.
answered Mar 13, 2017 at 11:37