1
$\begingroup$

I have four time series $A,B,X,Y$. I found that somehow, $\rho( A, B ) = 0.9999$, $\rho( X, Y ) = 0.9999$, I have also validated that $\sigma_A = \sigma_X$, $\sigma_B = \sigma_Y$

In other words $A$ and $X$, $B$ and $Y$ are really almost just two pairs of near-identical time series.

However I observe that $\rho( A - B, X - Y) = 0.85$

[EDITED after Alex's answer] I defined a "modified correlation coefficients" $\phi(X, Y) = \frac{X \cdot Y}{ \sqrt{X \cdot X} \sqrt{Y \cdot Y } } $ here $\cdot$ means a dot product.

I still observe that $\phi( A, B ) = 0.9999$, $\phi( X, Y ) = 0.9999$. so I think I can really conlude that they are near-identical time series ... But I still observe that $\rho( A - B, X - Y) = 0.85$. $\phi( A - B, X - Y) = 0.87$ slightly higher but not much.

What could be the cause that there is such big drop in correlation (or modified correlation) after I subtract them?

$\endgroup$
2
  • $\begingroup$ Hi @Mattt Frank, what do you mean by $\rho( A - B ) = \rho( X - Y) = 0.85$? Do you mean $\rho( A - B , X - Y) = 0.85$ $\endgroup$
    – Alex J
    Feb 22 at 1:55
  • $\begingroup$ Given any possible correlation coefficient $-1\lt r\lt 1,$ you can construct an example like this. Simply find two variables $\epsilon$ and $\delta$ with correlation $r,$ start with $A$ and $X,$ and define $B = u^\prime A+u\epsilon$ and $Y=v^\prime X+v\delta$ for suitably small values of $u$ and $v.$ (The coefficients $u^\prime$ and $v^\prime$ will both be close to $1$ and are included to guarantee equality of variances.) You can check that as $u$ and $v$ shrink to $0,$ the correlation of $A-B$ with $X-Y$ approaches $r$ in the limit. $\endgroup$
    – whuber
    Feb 22 at 20:20

2 Answers 2

2
$\begingroup$

It's worth noticing what $A-B$ and $X-Y$ refer to. Each is essentially the random noise that differs between the two members of each pair. It's frankly quite surprising to me that the random noise of the first pair is so highly correlated with the random noise of the second pair. That suggests that any perturbations in the sync within pairs happens across pairs.

For example, let's say $A$ and $B$ are the sales of chocolate and vanilla ice cream at store 1, and $X$ and $Y$ are the sales of chocolate and vanilla ice cream at store 2, and let's say that chocolate and vanilla sales are extremely highly correlated. Let's also say that chocolate sales are slightly influenced by the presence of a commercial playing one day for chocolate ice cream. On days the commercial airs, chocolate sells a tiny bit better at both stores.

This would yield a scenario very similar to what you observe: $A$ (chocolate ice cream sales at store 1) and $B$ (vanilla ice cream sales at store 1) are highly correlated, and $X$ (chocolate ice cream sales at store 2) and $Y$ (vanilla ice cream sales at store 2) are highly correlated, but on the few days $A$ is higher than $B$, $X$ is also higher than $Y$, which leads to a high correlation between $A-B$ and $X-Y$.

In the absence of a commercial or other shock affecting both pairs of time series, you might expect $A-B$ and $X-Y$ to be completely uncorrelated. That is, sometimes $A$ is higher than $B$ and sometimes $X$ is higher than $Y$, but there doesn't have to be anything linking those two events. Such a high correlation suggests there is a linking of those two events.

$\endgroup$
1
$\begingroup$

Your second sentence $A$ and $X$, $B$ and $Y$ are really almost just two pairs of near-identical time series does not necessarily follow from the first statement.

Consider this example in R. I define the four time series that satisfy the conditions on $\rho(A,B)$, $\rho(X,Y)$, $\sigma_A=\sigma_X$, $\sigma_B=\sigma_Y$.

But if you look at what the four time series actually are, they're obviously not identical. $A$ and $B$ are increasing over time, but $X$ and $Y$ are decreasing. Remember that correlation quantifies how one variable changes with another, not absolutely what the variables are doing.

A <- seq(0, 100, by = 1)
B <- seq(0, 5, length.out = 101)
X <- seq(0, -100, by = -1)
Y <- seq(0, -5, length.out = 101)

cor(X, Y) # ~ 1 as required
cor(A, B) # ~ 1 as required

sd(A); sd(X) # same as required
sd(B); sd(Y) # same as required

cor(A-B, X-Y) # the sign has reversed
$\endgroup$
6
  • $\begingroup$ Thanks Alex. I edited my question to address your counter-example (which is very informative) $\endgroup$
    – Matt Frank
    Feb 22 at 2:24
  • $\begingroup$ Do you have a link somewhere about your "modified correlation coefficient"? It's not something I am familiar with. $\endgroup$
    – Alex J
    Feb 22 at 2:43
  • $\begingroup$ I kind of made that up to measure the "magnitude" problem you identified ... $\endgroup$
    – Matt Frank
    Feb 22 at 2:48
  • $\begingroup$ What's it supposed to be doing? Just asking because it's not immediately clear to me $\endgroup$
    – Alex J
    Feb 22 at 2:54
  • $\begingroup$ Re the magnitude thing - have you tried plotting the time series? That to me would be the quickest way to see if they are actually the same or not, and would probably provide some insight on any difference $\endgroup$
    – Alex J
    Feb 22 at 2:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.