I am trying to better understand statistical significance, effect sizes and the like.
I have a perception (perhaps its wrong) that even irrelevant regressors often become statistically significant in large samples. By irrelevant I mean that there is no subject-matter explanation why the regressor should be related to the dependent variable. Thus irrelevance in this post is a pure subject-matter concept and not a statistical one.
I know that a regressor will be statistically significant given a sufficiently large sample unless the population effect is exactly zero (as discussed here). Hence, an irrelevant regressor that appears statistically significant in a large sample has a non-zero effect size in population.
Questions:
- How come an irrelevant regressor turns out statistically significant?
- Should I look for subject-matter explanation (i.e. try to deny irrelevance) or is this a statistical phenomenon?
This is a continuation of a post where I was trying to clarify how to cure this effect. Meanwhile, here I am asking why it happens in the first place.