Should one control for non-confounders? The standard error of the variable of interest $x$ can be calculated as 
$$s.e.({\hat\beta_x})=\sqrt{VIF_x\frac{\sigma_\varepsilon^2}{nVar(x)}} $$
As usual, $\sigma_\varepsilon^2=\sum_i\varepsilon_i^2 $ is the variance of the regression error and $VIF_x$ is the variance inflation factor of $x$.
If one now controls for a second variable $z$ (which happens to be highly significant), one inevitably reduces the residuals $\varepsilon_i$ what leads in turn to a reduction of $\sigma_\varepsilon^2$. Because $z$ is not a confounder, $VIF_x$ remains unchanged. Ultimately, $s.e.(\hat\beta_x)$ would go down.
If my thinking is right, then one would try to control in large datasets (say, 100k observations) for as many highly significant control variables as possible. That is because the loss in degrees of freedom is negligible and the $p$-value of the variable of interest goes down.
Whether to control for non-confounders seems to be quite an important question to get right in applied statistics. I am therefore wondering whether my argument is correct or whether I got something wrong?
Best wishes,
Tom
 A: VIF doesn't give us too much intuition about causal modeling. If a confounder is highly correlated with the exposure and the outcome, you should adjust for it even when it reduces the power of the analysis. 
What you are talking about is precision variables. If a covariable predicts (or causes) the outcome of interest, and has no association with the predictor of interest, you should adjust for it. These "precision" variables just increase the power of the analysis. So their relevance in model selection applies more to the statistical aspects of the analysis rather than the scientific. 
Statistical significance is not the basis of selecting such covariables for adjustment in analyses. A covariable can have a statistically significant association with the outcome because it is a mediator or a collider, so adjustment for it biases analyses and reduces power. Alternately, a covariable can be an actual confounder despite having a non-statistically significant association with the outcome after adjustment for other variables. Covariables are chosen for adjustment by drawing Directed Acyclic Graphs (DAG) and ensuring the criterion for appropriate causal modeling are met.
