3
votes
Accepted
Don't understand why SelectKBest with Chi Square does not involve p-value
The documentation says
Select features according to the k highest scores.
which really explains the whole thing. The class is designed to select the $k$ highest scores, no more, no less.
The concept ...
2
votes
Accepted
How can a feature that when removed, does not affect the model's performance not be declared unimportant?
One immediate example which comes to my mind would be a case where predictors are highly co-linear. If you have two covariates which are highly correlated, removing one of them will improve the ...
2
votes
Accepted
How XGBoost chooses between two features that gives the same information?
Yes, this is what $0\%$ importance for X2 in the presence of X1 suggests.
Now, the "how" is somewhat open-ended but ...
1
vote
Accepted
Best practice for Post-Double Selection LASSO (pdslasso)
Let me first briefly summarize the setting: We have a scalar treatment variable $D_i$, a grouping variable $Z_i$ (driver of heterogeneity) and high-dimensional controls $X_i$. $X_i$ can be high-...
1
vote
Don't understand why SelectKBest with Chi Square does not involve p-value
Note that chi2 returns p values, but you don't even need the p value you just need the test statistic and degrees of freedom. From those two pieces of information ...
1
vote
How to tell if my features improve model performance?
This is what an out-of-sample test set reveals. In fact, machine learning tends not to care much about in-sample (“training set”) performance, since you can play connect-the-dots and memorize the data,...
1
vote
Why do the results of LASSO regression differ after removing uninformative variables in glmnet?
Define a covariate matrix $X \in \mathbb{R}^{n \times p}$ with columns $\{x_j\}$, where there are $n$ observations of $p$ covariates. Define the response $y \in \mathbb{R}^p$. Suppose we are ...
Only top scored, non community-wiki answers of a minimum length are eligible
Related Tags
feature-selection × 2240machine-learning × 617
regression × 356
classification × 286
r × 237
feature-engineering × 187
random-forest × 179
lasso × 162
cross-validation × 136
neural-networks × 115
logistic × 112
pca × 102
svm × 97
model-selection × 95
scikit-learn × 92
predictive-models × 91
importance × 89
correlation × 85
multiple-regression × 83
python × 75
clustering × 67
dimensionality-reduction × 66
regularization × 65
boosting × 61
time-series × 56