I'm using the LASSO method, in the problem of text classification (sentiment classification). The features I'm using are mainly Ngrams (every N consecutive words) and I'm using the LASSO specifically so that I can rank the features and extract the set of the significant Ngrams in the classification problem.
My question is about tuning the
alpha parameter in the scikitlearn model: I understand that as I set
alpha closer to 1, the number of features selected becomes fewer. So I wanted to ask:
- What is the best practice to select the number of the important features, hence alpha value (cross validation could be possible if I seek maximum score not model interpretation), but is there's something to measure the "minimum adequate number of features for the classification process"?
- If I decided that I would like to select only the top 1000 features and set alpha to return 1000 features with non zero coefficients. Would the LASSO method here differ from using normal linear regression and rank the top 1000 features?