svd adds value before an elastic net model? I learned that SVD eliminates redundancies. If you use an elastic net model, is it still greedy as stepwise models in general? or the fact that it has penalization factors reduces the greedy component?
If you are trying to predict  a continuous number and the predictors are words and their frequencies in each document, would it be a good idea to use first SVD and then something like the elastic net model?
Thanks
 A: The elastic net is not greedy like stepwise model, it considers all variables together and shrinks their effects, sometimes to zero (depending on the penalty). In other words, the stepwise can depend heavily on the order in which you do the selection, but elastic-net does not (up to numerical issues which are unavoidable) [Note that lasso also depends to some extent on the order, but not as badly as stepwise]. So general advice is that it's best to avoid stepwise.
Regarding SVD (you really mean PCA I guess), then it depends on how many variables and samples you have. Doing dimensionality reduction will always remove information from your data, by definition, but in some cases that reduction may be offset by being able to better model the reduced data than the original data. So it's a case of trying out and seeing if it works. You do lose the interpretability since your new variables are now linear combinations of your original variables.
PCA is one way to decompose your data, but since it's frequencies which are non-negative numbers you might want to try out other decompositions like non-negative matrix factorization (NMF) that impose such constraints.
