3 votes
Accepted

Don't understand why SelectKBest with Chi Square does not involve p-value

The documentation says Select features according to the k highest scores. which really explains the whole thing. The class is designed to select the $k$ highest scores, no more, no less. The concept ...
user avatar
  • 78.3k
2 votes
Accepted

How can a feature that when removed, does not affect the model's performance not be declared unimportant?

One immediate example which comes to my mind would be a case where predictors are highly co-linear. If you have two covariates which are highly correlated, removing one of them will improve the ...
user avatar
2 votes
Accepted

How XGBoost chooses between two features that gives the same information?

Yes, this is what $0\%$ importance for X2 in the presence of X1 suggests. Now, the "how" is somewhat open-ended but ...
user avatar
  • 35.3k
1 vote
Accepted

Best practice for Post-Double Selection LASSO (pdslasso)

Let me first briefly summarize the setting: We have a scalar treatment variable $D_i$, a grouping variable $Z_i$ (driver of heterogeneity) and high-dimensional controls $X_i$. $X_i$ can be high-...
user avatar
  • 123
1 vote

Don't understand why SelectKBest with Chi Square does not involve p-value

Note that chi2 returns p values, but you don't even need the p value you just need the test statistic and degrees of freedom. From those two pieces of information ...
user avatar
1 vote

How to tell if my features improve model performance?

This is what an out-of-sample test set reveals. In fact, machine learning tends not to care much about in-sample (“training set”) performance, since you can play connect-the-dots and memorize the data,...
user avatar
  • 31.4k
1 vote

Why do the results of LASSO regression differ after removing uninformative variables in glmnet?

Define a covariate matrix $X \in \mathbb{R}^{n \times p}$ with columns $\{x_j\}$, where there are $n$ observations of $p$ covariates. Define the response $y \in \mathbb{R}^p$. Suppose we are ...
user avatar
  • 556

Only top scored, non community-wiki answers of a minimum length are eligible