6
$\begingroup$

I am trying to calculate lagged Pearson correlation coefficient between time series. I am not interested in calculating cross-correlation. I want to calculate Pearson correlation coefficient because I want to use the correlation for prediction.

In general, when two variables are strongly correlated, we get a high correlation coefficient. We can fit a straight line in the scatter plot of the two variables and use it for predicting $y$ (based on $x$). In such a case, the variance explained by the fitted line would be equal to the square of the correlation coefficient.

But when the two variables are arranged in a certain lag and then Pearson correlation coefficient is calculated between them, can we still say that the variance explained will be equal to the square of the correlation coefficient? Can we use the best fit line from the lagged scatter plot for prediction?

Improve this question
$\endgroup$
5
  • $\begingroup$ I think it is more precise to say "Pearson correlation between lagged time series" rather than "lagged Pearson correlation between time series" -- because it one of the time series that gets lagged with respect to another, not the correlation coefficient. $\endgroup$
    – Richard Hardy
    Commented Mar 6, 2016 at 9:26
  • $\begingroup$ Thanks Richard for a thorough answer. I still want to make sure of one point though. So based on your answer, when we use cross correlation function such as 'xcorr' in matlab (using 'coeff' option), then the function is calculating Pearson correlation coefficient right? The matlab page does not provide the formula so I was not sure. $\endgroup$
    – Sagar Parajuli
    Commented Mar 6, 2016 at 23:59
  • $\begingroup$ I don't know the Matlab implementation, but for example in R cross correlation is Pearson correlation for lagged series. $\endgroup$
    – Richard Hardy
    Commented Mar 7, 2016 at 6:22
  • $\begingroup$ Do you know how are the blank cells treated while calculating correlation in R? Are they replaced by zeros as in cross correlation in matlab or they are replaced by NaNs? $\endgroup$
    – Sagar Parajuli
    Commented Mar 9, 2016 at 19:12
  • $\begingroup$ There can be no blank cells in R (regardless of what you are going to do later, such as calculate correlation). You may check how the data was read into an R object (data frame, for example) before calculating the correlation. The cor function itself can treat NA values differently; type ?cor in R for details. $\endgroup$
    – Richard Hardy
    Commented Mar 9, 2016 at 19:19

1 Answer 1

Reset to default
4
$\begingroup$

But when the two variables are arranged in a certain lag and then Pearson correlation coefficient is calculated between them, can we still say that the variance explained will be equal to the square of the correlation coefficient?

Yes, if you consider explaining the variance of $y_t$ by $x_{t-h}$ where $h$ is the lag order. (But not necessarily so if you consider explaining the variance of $y_t$ by $x_t$.)

Can we use the best fit line from the lagged scatter plot for prediction?

Yes. It is actually quite practical because you have the lagged values earlier than you have the contemporaneous values, hence it is natural to use lagged -- rather than contemporaneous -- values for prediction.

I am trying to calculate lagged Pearson correlation coefficient between time series. I am not interested in calculating cross-correlation.

Cross correlation is the Pearson correlation for lagged time series (when one series is lagged with respect to another.)

Also note that correlation is a natural measure for cross-sectional data where the observations can be assumed to be $i.i.d.$, but it is not that natural in the time series setting where there is time dependence between observations. For example, Pearson correlation is not very useful when applied on two series sharing a common deterministic or stochastic trend (see Spurious correlations website for some examples).

Improve this answer
$\endgroup$
5
  • $\begingroup$ I just checked the matlab implementation and confirmed that in matlab cross correlation is NOT the Pearson correlation as you said. So your answer "Cross correlation is the Pearson correlation for lagged time series (when one series is lagged with respect to another" is not correct. $\endgroup$
    – Sagar Parajuli
    Commented Mar 9, 2016 at 20:04
  • $\begingroup$ I would say that it is Matlab that uses poor terminology rather than that my answer is incorrect. But at least it's good to know how things are actually implemented in the different software packages. $\endgroup$
    – Richard Hardy
    Commented Mar 9, 2016 at 20:20
  • $\begingroup$ @SagarParajuli, I had to scroll down all the way in this site to find how Matlab defines cross correlation (in section "More about"). Indeed, it seems to be using poor terminology as it is calculating the empirical non-centered second cross-moment, which is not correlation but which could be covariance if the first moment of at least one of the series is zero. Please note that in my answer I am using the standard definition of cross correlation rather than the specific one that is used in Matlab. $\endgroup$
    – Richard Hardy
    Commented Mar 9, 2016 at 20:28
  • $\begingroup$ That makes sense but even then cross correlation can be calculated using 'Spearman correlation' as well so we should not presume 'Pearson correlation' in any case. May be that R calculate 'Pearson correlation' by default while calculating cross correlation as you said. $\endgroup$
    – Sagar Parajuli
    Commented Mar 9, 2016 at 20:57
  • $\begingroup$ Yes, that could be done. And R indeed does what I indicated. In my experience "correlation" used without a predicate indicates "Pearson correlation", while Spearman and Kendall versions are less frequently used and most of the time referred to explicitly. But better be precise than obscure. $\endgroup$
    – Richard Hardy
    Commented Mar 9, 2016 at 21:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.