# Help needed with correlating two datasets

First cross-validated question so please be gentle :o)

I have two datasets all gathered and managed in 'R'...

Dataset 1 - News Corpus. Contains 3,270 entries from the period 1/Apr/13 to 31/Mar/14. There are often multiple stories on any one day, and indeed days with no stories at all (which I believe makes for an incomplete time series and problems). The dataset structure is;

Date - (a date)
Domain - (a string) with 8 levels i.e. there are 8 web domains
DomainType - (a string) with 4 levels e.g. "other news" or "technology news"
Sentiment_Title - (a numeric) a score that currently sits in range -4:4
Sentiment_Description - (a numeric) a score that currently sits in range -6:7
Sentiment_Body - (a numeric) a score that currently sits in range -53:146
CCAT - (logical)
ECAT - (logical)
GCAT - (logical)
MCAT - (logical)


Dataset 2 - Bitcoin Market Data. 365 day time series of weighted price, volume and intra-day spread for four different exchanges.

The Problem What I really want to know is which features (if any) of dataset 1 (the corpus) are significantly related to the time series and how. I guess the time series also needs leads and lags applied to know which direction any relationship goes and how far away from the story publication date that relationship lays.

I have spent a couple of weeks applying the very basic stats knowledge I have to the task and have spent a couple of hours with a post-grad stats support group who also proved unable to find a method that could be readily applied.

I (we) looked at basic Pearson's and Spearman's, moved on to look at linear regression and generalised linear models and so far there appears to be issues with the residuals that makes the output bunkum apparently. I believe vector-autoregression could also be applied but we are way off into realms I just don't understand yet.

The Question Given the datasets (and, ideally R) can anyone suggest or indeed offer up an approach to solving my problem? Even better some simple explanation of how to interpret the results of any such approach.

• I do not have expires in this area, so I will only leave a suggestion: would you consider breaking this into several multivariate time series analyses? That is, use multivariate time series analysis (this must exist, also see the microarray gene expression literature for this) on each aspect of the bitcoin data set? This looks like it would only be 12 time series. Also, you could maybe look into canonical correlation analysis to at least explore what potential relationships there are. CCA would ignore the time series structure, but from an exploratory standpoint, could be useful. – Moose Jun 2 '15 at 10:42

Vector Autoregression (VAR) (not to be confused with Value at Risk, VaR) allows you to deal with your multiple observations on your bitcoin data, and will model lagged dependencies of volume on day $t$ on volume, spread and price on day $t-1$. So this seems like a start.