LASSO: k-fold Cross Validation of AR(p)-process

Question

To improve my intuition on shrinkage models, I want to "recode" the lasso by myself. However, I'm at the point, where I have to program the k-fold Cross-Validation. At my future application of the algorithm I will use time series models, especially (V)AR(p). Because of the time-ordering, the random k-fold- Validation" should cause into problems, right ? Regarding to that I've two questoins:

1) How the glmnet package is dealing with the cv of AR models

2) Do you know a way some sources, that show me how I can deal with it ?

I found this in the blog of hyndman

Answer 1

You need to use expanding window or rolling window, consider below example:

Lets say, you have a data-set of 100 obervations and you need at least 10 observations to fit a model, and you are estimating an AR(1) model. Then your algorithm will look like this:

FEATURES = X = [y_1, y_2, y_3, ...., y_99]

TARGET = Y = [y_2, y_3, y_4, ...., y_100]

CV_SCORES = []

FOR n in [10, 99) DO:

    train_X = [y_1, y_2, ..., y_n]

    train_Y = [y_2, y_3, ..., y_{n+1}]

    test_X = [y_{n+1}]

    test_Y = [y_{n+2}]

    fit your model to (train_X, train_Y)

    predict with test_X

    calculate the score with prediction and test_Y

    append the score to CV_SCORES
END

The methodology above is expanding window and it is time-series version of leave-one-out cross validation.