14
$\begingroup$

Consider the following graph:

twitter and trading volume

The red line (left axis) describes the trading volume of a certain stock. The blue line (right axis) describes the twitter message volume for that stock. For instance, on May 9 (05-09) about 1.100 million trades and 4.000 tweets were made.

I would like to calculate whether there is a correlation between the timeseries, either on the same day or with a lag - for instance: tweet volume correlates with trading volume one day later. I'm reading many articles who have made such analysis, for instance Correlating Financial Time Series with Micro-Blogging Activity, but they do not describe how such an analysis is made in practical terms. The following is stated in the article:

enter image description here

However, I have very little experience with statistical analysis and don't know how to execute this on the series that I have. I use SPSS (also known as PASW) and my question is: what are the steps to take to make such an analysis from the point where I have a datafile underlying the above image? Is such a test a default feature (and what is it called) and/or how could I else execute it?

Any help would be greatly appreciated :-)

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ You can calculate them... you just can't compare them to critical values unless the two series are bi-variate normal $\endgroup$
    – IrishStat
    Apr 29, 2012 at 21:57
  • $\begingroup$ I've pasted raw data here: pastebin.com/tZajRae9 Is there a way to tell whether the series are bi-variate normal? I would really appreciate your comment. $\endgroup$
    – Pr0no
    Apr 29, 2012 at 22:34
  • $\begingroup$ After detecting the Outliers/Level shifts in each of the series the resultant adjusted series exhibited an AR(1) model. After incorporating not only the Outlier / level Shift adjustment AND the empirically identified AR (1) both noise series were free of auto-correlation (within structure). A cross-correlation of this two surrogate series indicated no substantive cross-correlation (among structure ) thus the number of tweets doesn't appear to help the prediction of volume. $\endgroup$
    – IrishStat
    Apr 30, 2012 at 14:54

2 Answers 2

Reset to default
6
+25
$\begingroup$

Two check for bivariate normality check three things:

  1. check if first series of observations is marginally normal,
  2. check if second series of observations is marginally normal,
  3. regress on one the other and check if residuals are normal.

To check normality at each of these steps, use normal q-q plots or you can use any normality hypothesis test.

Or alternatively you could check if every possible linear combination (real coefficients) of the two series is marginally normal. That would probably be difficult, though.

Edit: (6 years later) I'll keep the above for posterity, but note I have a more recent answer to a similar question here.

Improve this answer
$\endgroup$
7
  • $\begingroup$ I have taken steps 1 and 2 and came up with the following boxplots: i.imgur.com/SDOTE.png Except for the 3 to 5 outlier observations, they look marginally normal. However, the Sig. value for the Shapiro-Wilk Test is 0.000, which would indicate a significant deviation from normality. With outliers removed, Shapiro Wilk Sig. is 0.201 for tweets and 0.004 for trades. Does this indicate no correlating is possible? Also, this is a timeseries - deleting outliers means deleting days within the researched timeframe. Is this an accepted practice? $\endgroup$
    – Pr0no
    Apr 30, 2012 at 8:37
  • $\begingroup$ I also made a p-p plot for step 3. Or at least, in my interpretation this is what I need (a linear regression with normal probability plot): i.imgur.com/EZ3Ic.png Any comments? $\endgroup$
    – Pr0no
    Apr 30, 2012 at 8:45
  • $\begingroup$ The marginal distributions don't look normal. There is a small section on inference on the wikipedia page link. Removing outliers is generally not a good idea. Maybe bootstrap a confidence interval. $\endgroup$
    – Taylor
    May 2, 2012 at 21:09
  • 1
    $\begingroup$ The question is about correlation -- but the answer is about normality. The answer is upvoted multiple times and accepted. What am I missing here?.. $\endgroup$
    – Richard Hardy
    May 19, 2015 at 12:59
  • $\begingroup$ A bivariate normal distribution is the simplest model that motivates/justifies using Pearson correlation. $\endgroup$
    – Taylor
    May 20, 2015 at 14:02
11
$\begingroup$

The correlation coefficient between time series is useless. See CORRELATION COEFFICIENT - Critical values for Testing Significance. This was first pointed out by U. Yule in 1926 Yule, G.U, 1926, "Why do we sometimes get nonsense correlations between time series? A study in sampling and the nature of time series", Journal of the Royal Statistical Society 89, 1–64. You might want to google "why do we get nonsense correlation" for more.

The reason for this is tests for correlation requite joint normality. Joint normality requires each series to be normal. Normality requires independence. To examine the relationship between time series please review Transfer Function Identification in any good time series book like Time Series Analysis: Univariate and Multivariate Methods, by William W.S. Wei, David P. Reilly.

Challenge Answer

In terms of an answer to your challenge. It is well known, by a few (Yule, G.U, 1926) that correlating two time series can be flawed particularly if either series is affected by pulses/level shifts/seasonal pulses and/or local time trends. That being the case I would take each of the series SEPARATELY and identify the ARIMA structure and any pulses/level shifts/seasonal pulses and/or local time trends that might apply and create an error process.

With two clean error processes, one for each of the two original series, I would compute the cross correlation which could then be used to measure the degree of association above and beyond the auto-correlative structure within each series. This solution is appropriately called the double pre-whitening approach.

See:

Improve this answer
$\endgroup$
10
  • $\begingroup$ Thanks for your reply. But are you then saying that by definition, a.o. the paper I referred to, has no value? Secondly, does this mean that by definition two series can never be correlated where cthe correlation has meaning? $\endgroup$
    – Pr0no
    Apr 20, 2012 at 21:15
  • 3
    $\begingroup$ The correlation can be computed as that is simple arithmetic. What can't be computed( easily) is the probability that the correlation is statistically significant. Think back to the first time you were introduced to the correlation coefficient. It was in the context of N independent samples where two characteristics/values were computed for each of the N independent samples and the joint density was bivariate normal. $\endgroup$
    – IrishStat
    Apr 20, 2012 at 23:07
  • 1
    $\begingroup$ Why does it require joint normality, and not just the same (symmetric?) distribution? i.e. wouldn't joint uniformity also work? $\endgroup$
    – naught101
    Apr 23, 2012 at 23:27
  • 1
    $\begingroup$ @NAUGHT101 . Critical values for the correlation coefficient are available under the assumption of joint-normality and undefined else wise. $\endgroup$
    – IrishStat
    Apr 29, 2012 at 21:56
  • $\begingroup$ @IrishStat Thank you for your editted answer. It is appreciated. For normality testing, please see i.imgur.com/SDOTE.png for q-q plots of the separate variables. After outliers are removed, a p-p plot, from what I understand which measures joint-normlaity, looks like this i.imgur.com/EZ3Ic.png Any comments? $\endgroup$
    – Pr0no
    Apr 30, 2012 at 8:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.