2
$\begingroup$

I have two variables, say A and B. Pearson's correlation between the raw data of A and B is positive (not significant). But after taking the logarithm of one variable, say correlation(A, log(B)), the sign of the correlation becomes negative. How is this possible?

$\endgroup$
  • $\begingroup$ You can answer this yourself by examining the scatterplots of the pairwise variables. $\endgroup$ – Mike Hunter Mar 26 '16 at 14:44
  • 2
    $\begingroup$ The logarithm is a monotonic transformation. I don't think this should happen. How large is the positive & negative correlation coefficients? Are they both almost 0? Can you provide a scatterplot of your data? $\endgroup$ – gung Mar 26 '16 at 14:55
  • 1
    $\begingroup$ One way this could be possible is that the positive sign is largely a side-effect of an outlier that is subdued by the transformation. Note that changes of sign sound dramatic, but a change between very small positive and very small negative could be quite unimportant $\endgroup$ – Nick Cox Mar 26 '16 at 17:48
  • $\begingroup$ @gung I've constructed a pair of random variables $A$ and $B$ in my answer that satisfy $\rho(A, B)\approx 1$ and $\rho(A, \log(B)) < 0$. The basic trick I used was assigning $B$ values very close to 0 when $A$ takes high values, meaning $log(B)$ takes values as close as I want to $-\infty$. $\endgroup$ – josliber Mar 26 '16 at 17:53
  • $\begingroup$ I saw that, @josliber, +1. But I think the data need that special setup. I think it will be more typical for a monotonic transformation to yield a correlation with the same sign. $\endgroup$ – gung Mar 26 '16 at 18:27
3
$\begingroup$

Base Result

Random variables $A$ and $B$ can have a positive correlation but a negative correlation when one is logged. An easy way to see this is to consider two random variables that have positive correlation but where $B$ takes values arbitrarily close to 0 when $A$ is large; since $\log(B)$ will then take values arbitrarily close to $-\infty$ as $A$ gets large, $A$ and $\log(B)$ will be negatively correlated.

As a worked example, let $A$ and $B$ be discrete random variables whose values always coincide; for constants $x, y, \epsilon, \delta > 0$, let $(A, B)$ take value $(0, x)$ with probability $\frac{1-\delta}{2}$, let $(A, B)$ take value $(1, y)$ with probability $\frac{1-\delta}{2}$, and let $(A, B)$ take value $(2, \epsilon)$ with probability $\delta$. Then arithmetic shows:

\begin{align*} \newcommand{\cov}{{\rm cov}} \cov(A, B) &= -\frac{(1-\delta)(1+3\delta)}{4}x + \frac{(1-\delta)(1-3\delta)}{4}y + \frac{3\delta-3\delta^2}{2}\epsilon\\[7pt] \cov(A, \log(B)) &= -\frac{(1-\delta)(1+3\delta)}{4}\log x + \frac{(1-\delta)(1-3\delta)}{4}\log y + \frac{3\delta-3\delta^2}{2}\log \epsilon \end{align*}

Setting $x=1, y=2, \delta=0.01, \epsilon=e^{-100}$ yields the desired result:

\begin{align*} \cov(A, B) &\approx 0.2252 \\ \cov(A, \log(B)) &\approx -2.8036 \end{align*}

Obtaining $\rho(A, B)\approx 1$ and $\rho(A, \log(B)) < 0$

We can construct a pair of random variables $A$ and $B$ with $\rho(A, B)\approx 1$ and $\rho(A, \log(B)) < 0$ using the above example with $0 < x < y$, setting $\epsilon = e^{-1/\delta^2}$ and taking the limit as $\delta\rightarrow 0$. Basically we are selecting very small $\epsilon$ and $\delta$, but $\epsilon$ is getting small much faster than $\delta$.

We have the following results from the equations above:

\begin{align*} \cov(A, B) &\xrightarrow[\delta\rightarrow 0]{} \frac{y-x}{4}\\[7pt] \cov(A, \log(B)) &= \frac{-3}{2\delta} + o(\frac{1}{\delta}) \end{align*}

As a result, we see that $\cov(A, \log(B)) < 0$ as $\delta\rightarrow 0$ and therefore $\rho(A, \log(B)) < 0$ (it approaches 0 from below with the selected $\epsilon$ and $\delta$). To calculate the correlation between $A$ and $B$, we need to compute the standard deviations of the random variables:

\begin{align*} \sigma(A) &= \sqrt{\frac{1+8\delta-9\delta^2}{4}} \\[7pt] \sigma(B) &= \sqrt{\frac{1-\delta^2}{4}(x^2+y^2) + \frac{(1-\delta)^2}{2}xy - 2(x+y)\frac{1-\delta}{2}\delta\epsilon + \epsilon^2\delta(1-\delta)} \end{align*}

From these equations, we obtain $\sigma(A) \xrightarrow[\delta\rightarrow 0]{} 0.5$, $\sigma(B) \xrightarrow[\delta\rightarrow 0]{} \frac{y-x}{2}$, and therefore $\rho(A, B) \xrightarrow[\delta\rightarrow 0]{} 1$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.