3
$\begingroup$

So I am wondering if there usually is any "correction" being made to p-Values (of single predictors) if you add more predictors to the model? Does anybody know how SPSS handles this?

I am well aware that change of a predictors p-value after including another variable could occur due to 1) the predictors effect (regression coefficient) having changed and 2) because the sample size has decreased (if some of the previously included cases have a missing value at the newly included predictor and we use listwise exclusion). But could a change also be due to 3) the model taking into account that when I test a higher number of predictors, the likeliness of one attaining an effect of a given strength rises?

$\endgroup$
  • $\begingroup$ if you add more predictors to the model? I do not think that it should affect p--value! Why - correction is invoked. $\endgroup$ – Subhash C. Davar Jul 24 '19 at 14:31
4
$\begingroup$

There are two things to consider here.

First, the p value reported for an individual coefficient in a standard linear regression is based on a t-test. As that test uses an estimate of the standard error taken from the data, the reliability of the standard error estimate depends on both the number of observations and the number of parameters that are estimated. The degrees of freedom in the t-test decrease as the number of estimated parameters increases: $(N-P)$ for $P$ parameters estimated from $N$ observations. That's the level of correction for numbers of parameters in the p-values reported by standard statistical software from the t-tests for regression coefficients. Note that if the number of parameters estimated is much smaller than the number of observations, this adjustment won't make much of a difference.

Second, which I take to be the basis of your question, is the matter of multiple comparisons, the idea that the more significance tests you perform the more likely you are to find one nominally "significant" result simply by chance: a test at p < 0.05 when a null hypothesis is true will be a false positive in 1 out of 20 comparisons. Standard statistical software and published tables of estimated regression coefficients typically do not correct coefficient p -values for multiple comparisons. See the discusssion on this page for some thoughts.

One exception is software used for big-data analyses like gene-expression studies, which builds in important corrections for multiple comparisons. If you tried to rule out any false-positive results among the thousands of nominally "significant" coefficients you would necessarily miss a lot of true positives. In such situations the false discovery rate, the fraction of "significant" findings likely to be false, is typically controlled. See this page among many others on this site.

$\endgroup$
  • $\begingroup$ the reliability of the standard error estimate depends on both the number of observations and the number of parameters that are estimated. It is somehow ambigous and incorrect. please check whether it needs to be there. $\endgroup$ – Subhash C. Davar Jul 24 '19 at 14:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.