# Cross-validated cross-correlation

I am trying to build a regression model for the forecast of stock market returns. The regression takes about 100 variables as input and my training data consists of n=234 weekly data points. I am using a lasso regression for the regression + variable selection.

My problem is that before I can input the data into the regression I need to find the "optimal lag" time for it. The variables can be categorised into 4 groups and I have got data for the variables up to 113 weeks before the first point in time of my training data. To calculate the "optimal" lag time I tried following approach:

Take every variable for one categorie and apply a rolling window to the training data, which moves forward in quarters e.g. datapoints 1:13,14:26,... This gives 18 seperate quarters for the training data. For the first quarter calculate the correlation (spearman in this case) between the stock course and every variable in the categorie with a lag of 0. Then take the average of the absoulte correlation of all the variables for this quarter and enter it into a matrix at point [1,1]. Repeat this for every possible lack time up to 113 and then move on to the next quarter.

This results in a [114,18] matrix which includes the average absolute correlation for every lag in every quarter. Now I just take the average for every row in the matrix and the row with the highest average correlation should give me the most reliable lag for my data of that category.

Q: Has anyone seen a similar approach like this in literature before? Is there something I am missing? E.g. would it be smarter to just calculate the correlation over the whole trainingset at once and look for the optimal lag that way? As far as I am concerned that is the most common approach in literature, but it seems to me that it gives you a pretty biased result.

For some more insight you can find my code below:

Correlation.Maximiser = function(Stock, Category){

Result = matrix(nrow = 114,ncol =  18)

for(tmp in 1:18){
Start.Test=1+(tmp-1)*13
End.Test=13+(tmp-1)*13
Sample = Stock[Start.Test:End.Test]

for(i in 0:113){
int.low=101-i+13*tmp
int.high=113-i+13*tmp
NA.omitter = cor(Sample,Category[int.low:int.high,-1], method = "spearman")
NA.omitter[is.na(NA.omitter)] = 0 #some Variables have a lot of 0 values so that it can happen that they are only zero in the quarter we look at which results in an NA
Result[i+1,tmp]=mean(abs(NA.omitter))
}
}
return(Result)
}


EDIT: I just realised that the comparison in categories makes no real sense, since some variables might have a positive while others might have a negativ correlation to the dependent variable. I tried to compensate that by averaging over the absolute correlation, but I realised that this would not penalise variables that change sign from quarter to quarter and such behavior would be really bad for the regression. In general my question stays the same though. Only now the procedure is applied to every variable on its own.

• Not an answer to your exact problem, but Viceira (2007) works with returns and time varying realised cross-correlations (see SSRN). It's been also published by the IJF in their special edition on Predictability of Financial Markets (2012). Sep 17, 2015 at 12:50
• It appears that you are assuming no outliers\pluses in your data. What you are trying to do is to determine the typical response which requires that the exceptional response to not play a role in computing the correlations. I have programmed a similar solution which I refer to as the"kitchen-sink approach" as the user is throwing everything into the model except the proverbial kitchen sink . Sep 17, 2015 at 14:33