I want to train a neural network for regression. $$\Bbb R^{2800} \rightarrow \Bbb R^{1}$$ The dimension of feature vectors is $2800$. The figure is an illustration of one of the feature vectors.

a feature vector

Since the dimension of the feature vectors is relatively large, and there might be redundancies, I want to do an automatic feature selection to decrease the dimension.

Does it make sense to do lasso regression for selecting features? My concern is lasso regression is a linear regression method, however, a neural network can be trained to approximate non-linear relationships.

If lasso regression is not legitimate for selecting features for neural network training, how can I reduce the dimensionality of the feature vectors?


Although this is not equivalent to choosing the features, if your training duration is not too long, it might worth employing L1 regularization on the neural network weights themselves. If you choose to implement lasso from the beginning, it'll do the selection by thinking of only linear relationships as you say; this may or may not work. You can also do dimensionality reduction beforehand. Some candidates worth trying are PCA, NMF, autoencoders...


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.