Variable selection question In regression where there are multiple predictors $x_1, x_2, \dots$ let's say our feature selection algorithm (lasso, forward stepwise, etc.) returns that $x_1$ is an important predictor, but our client says there's no way $x_1$ is important. What could be happening?
 A: There are several possibilities and without details it is impossible to say which is most likely.  Here are a few:


*

*You are observing a Type I error, i.e. there really is no relationship but due to chance the data observed shows a relationship.  With a single test (and all assumptions holding) this will happen with probability $\alpha$.  In multiple testing (stepwise procedures) it will happen more often.  Violated assumptions can also increase the risk of a type I error.

*You are missing a lurking variable that is related to both $y$ and $x_1$.  It could be true that $x_1$ does not cause y, but that something else causes both $y$ and $x_1$ and therefore there is a relationship when that something else is not accounted for.

*Data entry error.  A single outlier in both $x_1$ and $y$ will make them appear related using the least squares methodology.  Or $x_1$ is really a different variable that does relate to $y$, but is labeled wrong.

*Your client now has an opportunity to learn something new because despite their understanding, there really is an unexpected relationship.

*Something else.

*Some combination of the above.
