I'm dealing with a multi-output regression problem (~ 800 dependent variables, ~ 1300 observations). My current approach is to train a single model for each output. To select an "optimal" lambda I tried to use cv.glmnet, but the training fails as some targets occured only a few (e. g. 2-5) times in the past (the other instances are 0 for this target). If this is the case, I need to force these instances to be present in each fold (some kind of stratified sampling). Otherwise there's a high probability that there's no variation in y and cv.glmnet produces following error message:
Error in elnet(x, is.sparse, ix, jx, y, weights, offset, type.gaussian, : y is constant; gaussian glmnet fails at standardization step.
How can I make sure / control that those observations are in each fold if the number of instances for a specific target is small?