3
votes
Accepted
Don't understand why SelectKBest with Chi Square does not involve p-value
The documentation says
Select features according to the k highest scores.
which really explains the whole thing. The class is designed to select the $k$ highest scores, no more, no less.
The concept ...
- 78.3k
2
votes
Accepted
How can a feature that when removed, does not affect the model's performance not be declared unimportant?
One immediate example which comes to my mind would be a case where predictors are highly co-linear. If you have two covariates which are highly correlated, removing one of them will improve the ...
- 389
2
votes
Accepted
How XGBoost chooses between two features that gives the same information?
Yes, this is what $0\%$ importance for X2 in the presence of X1 suggests.
Now, the "how" is somewhat open-ended but ...
- 35.3k
1
vote
Accepted
Best practice for Post-Double Selection LASSO (pdslasso)
Let me first briefly summarize the setting: We have a scalar treatment variable $D_i$, a grouping variable $Z_i$ (driver of heterogeneity) and high-dimensional controls $X_i$. $X_i$ can be high-...
- 123
1
vote
Don't understand why SelectKBest with Chi Square does not involve p-value
Note that chi2 returns p values, but you don't even need the p value you just need the test statistic and degrees of freedom. From those two pieces of information ...
- 25.8k
1
vote
How to tell if my features improve model performance?
This is what an out-of-sample test set reveals. In fact, machine learning tends not to care much about in-sample (“training set”) performance, since you can play connect-the-dots and memorize the data,...
- 31.4k
1
vote
Why do the results of LASSO regression differ after removing uninformative variables in glmnet?
Define a covariate matrix $X \in \mathbb{R}^{n \times p}$ with columns $\{x_j\}$, where there are $n$ observations of $p$ covariates. Define the response $y \in \mathbb{R}^p$. Suppose we are ...
- 556
Only top scored, non community-wiki answers of a minimum length are eligible