Simple distance measure for financial time series

Question

I have a large quantity of financial trading systems that I believe are highly duplicative, meaning that I believe a large number of the trading systems are essentially the same thing.

I am looking for a way to measure the differences between each of the trading systems. One simple way to do this is correlation of daily returns, although that limits me to linear relationships and a correlation matrix of 100k x 100k, which is not really useful.

What methods can I use to measure differences between the time series and then to cluster them together, with the eventual goal of removing the trading systems that are highly similar?

I'm new to this, so if I'm leaving out relevant information please let me know and I will modify this question.

Thanks.

Sample data is below:

StratID SystemID Date   Daily Simple Return
1   1   1/25/2011   0.04
1   1   1/26/2011   0.49
1   1   1/27/2011   -0.02
1   1   1/28/2011   0.76
1   1   1/31/2011   0.61
1   2   1/3/2011    1.37
1   2   1/4/2011    -0.02
1   2   1/5/2011    -0.52
1   2   1/6/2011    0.16
1   2   1/7/2011    0.85
1   2   1/10/2011   -0.14

Answer 1

Consider calculating the squared difference of each daily return and taking the mean over all returns (mean square error). You could conisder each daily return to be an axis in a high dimensional space and user standard clustering techniques, e.g. k-means is the easiest to understand and implement and it may be sufficient for what you want although I was advised at one time that k-means may not be a good choice for high dimensional spaces (but I have no data to back this up - I personally would try it as a first step).

Answer 2

You could take a look at cluster analysis. Essentially treat each strategy+system as an object and your goal is to cluster objects that are similar to one another in the same cluster.

A similarity metric that you could use is that of distance using the returns that a given strategy+system would give. Thus, distance between two objects would be the euclidean distance (or equivalently the absolute difference) between the returns corresponding to the two objects.

As a first-cut you may want to use hierarchical clustering. The wiki (see link) has not only has a description of how this clustering approach works but also some suggestions regarding procedures to use using various software tools.