6
$\begingroup$

I have a task which is related to finding correlations between time series. I have two financial time series given, which contain daily interest rate offers of two financial contributors and I want to find correlations (over time) between these quotes. The frequency and length is equal. The shape of the two time series is very similiar and looks like a upside down V. (upward trend followed by a downward trend)

My background in statistics, and particularly in time series analysis, is not very distinct. I know a few basics because I study mathematics but not more. I use the software R. My first approach was to calculate the Pearson correlation coefficient, but then I read a few topics about spurious correlation and fake correlations in trending time series so this could be not appropriate. Furthermore I read a few topics how to handle such problems, but I am still not sure how to solve my problem with my knowledge.

I would start as follows:

1) Use first differences or link relatives (which I found here: http://svds.com/avoiding-common-mistakes-with-time-series/) instead of absolute interest rates.

2) The hope is to get weak-stationary series so that I can calculate correlation coefficients (Pearson/Spearman) and cross correlation for different lags. Am I getting this right?

Would this course of action be appropriate to get meaningful results? How should I go on to solve the task? Im not looking for the perfect scientific solution, but I want to do a meaningful and solid analysis. Many thanks in advance!

Improve this question
$\endgroup$
5
  • $\begingroup$ Rank correlation (spearman / kendall) on the first differences should be fine! You can also visualize the scatterplot of the ranks (also known as empirical copula) to see if the relation is more or less comonotonic. $\endgroup$
    – mic
    Commented Jan 21, 2017 at 16:24
  • $\begingroup$ I got some results now. Correlation with spearman is ~0.5 and with Pearson ~0.8. How should I interpret these numbers? Intuition says that there must be a close correlation between the two time series. $\endgroup$
    – do-math
    Commented Jan 22, 2017 at 10:23
  • $\begingroup$ Can you display the scatterplot of the ranks? It would help to understand what is going on... Usually, on financial time series data, I obtain a Pearson correlation which is less than the Spearman one (due to heavy-tails / some strong deviations / outliers). $\endgroup$
    – mic
    Commented Jan 22, 2017 at 11:56
  • $\begingroup$ I do not really know how to display the scatterplot of ranks. Here is an usual scatterplot: !Scatterplot. Furthermore I have another question: Can I also use relative changes instead of first differences like described in my main post citation? $\endgroup$
    – do-math
    Commented Jan 22, 2017 at 13:19
  • $\begingroup$ It is rather easy: instead of display the point (x_i,y_i), you display the point ( rank(x_i), rank(y_i)), where rank is the function that takes as input x_i, and outputs the rank of x_i amongst x_1,x_2,...x_n. That is, if x_i is the jth biggest value of all values of x, and y_i is the kth biggest value of all values of y, the you plot (j,k). If you do that, it does not matter to use first differences or relative changes (returns), or log-returns since these results will be invariant thanks to this transform! $\endgroup$
    – mic
    Commented Jan 22, 2017 at 14:28

2 Answers 2

Reset to default
3
$\begingroup$

Find correlation between two time series. Theory and practice (R) discusses my road-map which is quite consistent with the very clear presentation that you cited http://svds.com/avoiding-common-mistakes-with-time-series/ . The simple though profound idea is that to identify the intra-relationship one needs to adjust for any inter-relationship in the candidate X. I strongly suggest reading the seminal article on this subject https://www.jstor.org/stable/2341482?seq=1#page_scan_tab_contents . If you have problems obtaining it please feel free to contact me and I will try and help you.

Finally care must be taken to deal with possible anomalies as their untreated presence induces obfuscation yielding a downwards bias in statistical tests of significance much like a pebble affecting your glasses which will affect your vision.

I wonder about Google's econometricians/statisticians scholarship http://people.ischool.berkeley.edu/~hal/Papers/2015/primer.pdf as they continue to apply ordinary correlation tests (GOOGLE CORRELATE ; GOOGLE TRENDS) to time series data where angels fear to thread suggesting search processes based upon the naive assumption of no intra-correlation which is functionally equivalent to the assumption when analyzing cross-sectional data when trying to identify/short list significant predictors.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks a lot for your informative answer! I am trying to understand it. But would you agree with mic's comment that my ideas of action (I am trying to keep the analysis simple, because of my few knowledge and not much time left) at least give me some analysis which is no nonsense and delivers meaningful results? $\endgroup$
    – do-math
    Commented Jan 22, 2017 at 10:20
  • $\begingroup$ I an not a particular fan of non-parametric methods when parametric methods are available . In short there could be lag effects of an important series that would not be picked up by simply looking at contemporaneous statistics. $\endgroup$
    – IrishStat
    Commented Jan 22, 2017 at 15:00
1
$\begingroup$

To complete my comments, here is a way to display the scatterplot of the ranks:

library(MASS)

nsample = 500
mu <- c(0,0)

#high correlation: 0.9
correl = 0.8

sigma = matrix(c(1, correl, correl,1),2)
sample <- mvrnorm(nsample, mu = mu, Sigma = sigma )
x = sample[,1]
y = sample[,2]

rkx = rank(x)
rky = rank(y)

plot(rkx,rky)

#low correlation: 0.2
correl = 0.2

sigma = matrix(c(1, correl, correl,1),2)
sample <- mvrnorm(nsample, mu = mu, Sigma = sigma )
x = sample[,1]
y = sample[,2]

rkx = rank(x)
rky = rank(y)

plot(rkx,rky)

Higher the correlation, closer to the diagonal lie the points of your scatterplot. You will obtain the same whether using 1st diff or log-returns (invariance by increasing transformation of your variables).

What is Spearman correlation? It is just Pearson correlation applied to this scatterplot instead of the original data, that is the best linear fit to the ranks.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you very much. Here is the scatterplot , it looks strange(Pearson gives 0,81 and Spearman 0,39: [![Rplot.jpg](s23.postimg.org/lmex0gva3/Rplot.jpg] $\endgroup$
    – do-math
    Commented Jan 22, 2017 at 15:47
  • $\begingroup$ This means that your variables are not continuous. For financial time series, it often happens when the assets are illiquid. I bet that you have many 0s (0 diff or 0 returns) in your data! Basically, the sampling frequency of your series is too high. If you want to make the variables more continuous, I would suggest to downsample it, for example taking weekly variations. $\endgroup$
    – mic
    Commented Jan 22, 2017 at 15:56
  • $\begingroup$ Unfortunately the task is to do the analysis on a daily basis with this sample size. Is my analysis with this data not meaningful? The value I got from Pearson seems to be good. $\endgroup$
    – do-math
    Commented Jan 22, 2017 at 16:04
  • $\begingroup$ Well, for me your Pearson value is absolutely not relevant! This just so because you have lots of small (even zero values) and some big values which happen to appear at the same time. Therefore, the best line is quite easy to find (just pass through the extreme values)... A Pearson correlation of 0.8 would correspond to the scatterplot you would obtain with the R code I pasted below. This is not at all what you get! Plus, you obtain some values in the upper-left corner, and down-right one. Basically, it is not likely to obtain them with linear correlation of 0.8. $\endgroup$
    – mic
    Commented Jan 22, 2017 at 16:14
  • $\begingroup$ Your dependence is not a standard linear or co-monotonic one. In this case using either linear (Pearson) or comonotonic (Spearman, Kendall) relationships make little sense. Either (i) get more accurate data, (ii) use weekly returns, (iii) report the 'weird' scatterplot. But giving your 0.4 Spearman, or 0.8 Pearson is VERY misleading. $\endgroup$
    – mic
    Commented Jan 22, 2017 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.