# Interpretation of high p value of a coefficient in linear regression [duplicate]

In a medical article, the multiple linear regression model included some coefficients with low p values but also some with p>0.05. How can I interpret these high p values and should they be be included in the model?

• You'd interpret it as any other $p$-value. A high $p$-value means that the value of the test statistic was not surprising assuming the null hypothesis was true (and all background assumptions hold). The second question is not answerable without more information. A good start would be for you to tell us what the goal of the model was and why you might think variables with a high $p$-value should be excluded from it. Commented Apr 23 at 19:31
• P-values are notoriously difficult to grasp, but luckily there are many posts on Cross Validated already (e.g., this one) trying their best to explain. Commented Apr 23 at 20:35
• Before interpreting any p-values, significant or otherwise, read about the Table 2 fallacy: Westreich D, Greenland S. The table 2 fallacy: presenting and interpreting confounder and modifier coefficients. Am J Epidemiol. 2013 Feb 15;177(4):292-8. doi: 10.1093/aje/kws412. Epub 2013 Jan 30. PMID: 23371353; PMCID: PMC3626058. Commented Apr 24 at 5:12
• @dariober these type of questions turn up a lot and it should be statedore clearly what exactly is the point of the question and how it is different from a general explanation about p-values. Currently it reads more like 'explain me what p-values mean?' and as a consequence this question is attracting already 4 answers of various types. I don't think it is useful for the website if we get many people answering different questions on a single question. Commented Apr 25 at 8:28
• @dariober the "should they be be included in the model" question is a bit of a side track and has it's own duplicates. Commented Apr 25 at 8:32

I sort of disagree (respectfully) that p-values could be seen as a measure of the impact of adding that variable in the goodness-of-fit of that model.

From my understanding, p-value is a function of:

• the true effect size within the data generation process
• the sample size of the model
• the variance of the response variable

Embedded in the notion of p-value there is the ability to detect such effect sizes (by effect size I mean the parameter trying to be estimated). So when a p-value is large, in my mind, I can think of the following reasons why:

• the true effect size is small
• my sample size can't detect the effect size (be it large or small)
• there is too much variance (or too little) to estimate the effect size (be it large or small)

All of those are backed by my experience when:

• you standardize your variables, many estimates have a significant p-value
• when you have a large sample size, everything is significant, sort of deeming it useless to look into p-values in those instances

Let me exemplify those thoughts with some simulations:

Here is an example of a case with:

• betas with small effect size
• low sample size
• high variance in response
library(dplyr)
N <- 50
variables <- 7
set.seed(1)
betas <- rnorm(variables, mean = 0.01, sd = 0.5)
frame <- as_tibble(replicate(variables, runif(N,-1,1)))
y <-  5 + as.matrix(frame) %*% betas + rnorm(N, 0, 10)
model_frame <- bind_cols( frame, as_tibble( y) %>% rename(y = V1))
lm(y ~ . , data=model_frame) %>% summary
#>
#> Call:
#> lm(formula = y ~ ., data = model_frame)
#>
#> Residuals:
#>      Min       1Q   Median       3Q      Max
#> -28.0209  -5.2780   0.0665   6.3169  17.9652
#>
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)
#> (Intercept)   3.6598     1.8640   1.963   0.0562 .
#> V1           -1.8986     3.3324  -0.570   0.5719
#> V2           -3.3655     3.0185  -1.115   0.2712
#> V3           -1.0679     3.3015  -0.323   0.7480
#> V4           -0.3517     2.9312  -0.120   0.9051
#> V5            0.8987     3.3845   0.266   0.7919
#> V6           -4.0196     3.1118  -1.292   0.2035
#> V7           -4.3301     2.8874  -1.500   0.1412
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 11.15 on 42 degrees of freedom
#> Multiple R-squared:  0.1123, Adjusted R-squared:  -0.03568
#> F-statistic: 0.7588 on 7 and 42 DF,  p-value: 0.6244


Here is an example of a case with:

• betas with small effect size
• high sample size
• high variance in response
N <- 100000
variables <- 7
set.seed(1)
betas <- rnorm(variables, mean = 0.01, sd = 0.5)
frame <- as_tibble(replicate(variables, runif(N,-1,1)))
y <-  5 + as.matrix(frame) %*% betas + rnorm(N, 0, 10)
model_frame <- bind_cols( frame, as_tibble( y) %>% rename(y = V1))
lm(y ~ . , data=model_frame) %>% summary
#>
#> Call:
#> lm(formula = y ~ ., data = model_frame)
#>
#> Residuals:
#>     Min      1Q  Median      3Q     Max
#> -47.416  -6.744   0.025   6.754  41.780
#>
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)
#> (Intercept)  5.03115    0.03155 159.446  < 2e-16 ***
#> V1          -0.30698    0.05451  -5.632 1.79e-08 ***
#> V2           0.04857    0.05471   0.888   0.3747
#> V3          -0.44610    0.05473  -8.151 3.66e-16 ***
#> V4           0.87422    0.05462  16.006  < 2e-16 ***
#> V5           0.13834    0.05474   2.527   0.0115 *
#> V6          -0.41664    0.05478  -7.606 2.85e-14 ***
#> V7           0.28010    0.05458   5.132 2.87e-07 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 9.978 on 99992 degrees of freedom
#> Multiple R-squared:  0.00443,    Adjusted R-squared:  0.004361
#> F-statistic: 63.57 on 7 and 99992 DF,  p-value: < 2.2e-16


Here is an example of a case with:

• betas with small effect size
• low sample size
• low variance in response
N <- 50
variables <- 7
set.seed(1)
betas <- rnorm(variables, mean = 0.01, sd = 0.5)
frame <- as_tibble(replicate(variables, runif(N,-1,1)))
y <-  5 + as.matrix(frame) %*% betas + rnorm(N, 0, 0.1)
model_frame <- bind_cols( frame, as_tibble( y) %>% rename(y = V1))
lm(y ~ . , data=model_frame) %>% summary
#>
#> Call:
#> lm(formula = y ~ ., data = model_frame)
#>
#> Residuals:
#>       Min        1Q    Median        3Q       Max
#> -0.280209 -0.052780  0.000665  0.063169  0.179652
#>
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)
#> (Intercept)  4.98660    0.01864 267.523  < 2e-16 ***
#> V1          -0.31918    0.03332  -9.578 4.00e-12 ***
#> V2           0.06715    0.03019   2.225   0.0315 *
#> V3          -0.41441    0.03302 -12.552 8.38e-16 ***
#> V4           0.79605    0.02931  27.157  < 2e-16 ***
#> V5           0.18199    0.03384   5.377 3.10e-06 ***
#> V6          -0.43643    0.03112 -14.025  < 2e-16 ***
#> V7           0.20788    0.02887   7.199 7.55e-09 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 0.1115 on 42 degrees of freedom
#> Multiple R-squared:  0.9608, Adjusted R-squared:  0.9543
#> F-statistic: 147.2 on 7 and 42 DF,  p-value: < 2.2e-16


Here is an example of a case with:

• betas with high effect size
• low sample size
• highish variance in response
N <- 50
variables <- 7
set.seed(1)
betas <- rnorm(variables, mean = 1.5, sd = 0.5)
frame <- as_tibble(replicate(variables, runif(N,-1,1)))
y <-  5 + as.matrix(frame) %*% betas + rnorm(N, 0, 1)
model_frame <- bind_cols( frame, as_tibble( y) %>% rename(y = V1))
lm(y ~ . , data=model_frame) %>% summary
#>
#> Call:
#> lm(formula = y ~ ., data = model_frame)
#>
#> Residuals:
#>      Min       1Q   Median       3Q      Max
#> -2.80209 -0.52780  0.00665  0.63169  1.79652
#>
#> Coefficients:
#>             Estimate Std. Error t value Pr(>|t|)
#> (Intercept)   4.8660     0.1864  26.105  < 2e-16 ***
#> V1            1.0272     0.3332   3.083 0.003618 **
#> V2            1.2451     0.3019   4.125 0.000171 ***
#> V3            1.0162     0.3302   3.078 0.003664 **
#> V4            2.1817     0.2931   7.443 3.40e-09 ***
#> V5            1.7371     0.3384   5.133 6.91e-06 ***
#> V6            0.7278     0.3112   2.339 0.024173 *
#> V7            1.2853     0.2887   4.451 6.18e-05 ***
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#>
#> Residual standard error: 1.115 on 42 degrees of freedom
#> Multiple R-squared:  0.8042, Adjusted R-squared:  0.7716
#> F-statistic: 24.65 on 7 and 42 DF,  p-value: 5.919e-13


### Concluding remarks

Of course, there is the need to define what I mean concerning low/high effect sizes, samples, and variances. But in the example I show, regardless of the p-value, every single variable should be included in the model, because they are a part of the data-generating process (and I, the researcher, know this in advance). If I didn't I would rely on:

• domain expertise
• causal diagrams
• experiment design

And only then would I judge the p-values as a way to gauge the appropriateness of having that variable in the model. Finally, how do I interpret high p-values? I attribute them to one of the following reasons (replicated from the start of this answer). Either:

• the true effect size is small (even possibly zero)
• my sample size can't detect the effect size (be it large or small)
• there is too much variance (or too little) to estimate the effect size (be it large or small)
• "when you have a large sample size, everything is significant, sort of deeming it useless to look into p-values in those instances" If a test resulting in low $p$-values (high power, consistent test) in a large sample is a problem you're using the wrong tool at any sample size. Commented Apr 24 at 16:21
• 100% agree. I just add this case to show how sample size can sway the p-value either way. Commented Apr 24 at 16:28

How to interpret? A high p-value means that including that variable in the model does little to improve the goodness-of-fit of the model to the data. In other words, the model that excludes that variable wouldn't fit much worse.

Should those variables (the ones associated with high p-values) be included in the model? Yes! It sounds sort of reasonable to make the model simpler by excluding the variables with high p-values. But this is not a good idea and it would lead to misleading results. Search this site for "stepwise regression" to learn why.

• P value as a goodness of fit measure? I don't think I've ever heard that interpretation before. Commented Apr 24 at 1:05
• @GuilhermeMarthe Of course, p-value is not a direct measure of goodness of fit. But if including a variable improves the goodness of fit, then the p-value would be small. So there is an indirect relation. Commented Apr 24 at 13:54

If you have data for a small number of people, say n=10, and you fit a model like:

height ~ sex + someVariableOfInterest


chances are that sex will have a high p-value because n=10 is not enough to discard the null hypothesis that height associates with sex. However, we do know that males are taller than females, on average. So should you include sex in the model?

I don't have a straight answer, but I would say in this case, yes, you should include it. How would you interpret the p-value for the sex coefficient? The interpretation is that given these data and model the coefficient of sex is compatible with a range of values which includes zero (but you know it is not zero).

And what if you have other variables that, like sex, you know associate with height? If you include them in the model, then you will not be able to say anything about someVariableOfInterest since with n=10 all the coefficients will be compatible with a large range of values (and likely including zero). Is this the right thing to do? I think so, since effectively you cannot exclude the possibility that someVariableOfInterest has virtually no effect and it is confounded with variables that do have an effect.

• The assertion "chances are that..." does not follow from the usual theory of p-values. Regardless of the sample size, the distribution of the p-value under a simple null $H_0$ for a continuous test statistic will be uniform while its distribution will favor small values under $H_A.$
– whuber
Commented Apr 25 at 12:09
• @whuber I'm not sure I understand your criticism. The p-value for sex with small n will be fairly often greater 0.05. If sex had no effect at all, p will be > 0.05 95% of the time upon repeated experiments. So I don't see a problem with chances are that sex will have a high p-value ... Commented Apr 25 at 12:50
• Perhaps this clarifies it: by "chances are" I don't mean more than 50% of the time. I mean "fairly often", almost as often as if the $H_0$ holds. Commented Apr 25 at 13:02
• Re your first comment: these chances do not depend on $n.$
– whuber
Commented Apr 25 at 15:07
• @whuber I'm afraid I don't get it: You measure height in 5 males and 5 females, do a t-test, repeat many times. Realistically you should get p > 0.05 in like 30% of the cases. Now do the same with 500 males and 500 females and almost never you get p > 0.05. Commented Apr 25 at 17:38

TL;DR A high p-value indicates that the coefficient is not large enough to be attributable to anything other than statistical noise - it is not statistically significant.

Underlying Assumptions By running a multiple linear regression model, you are making various assumptions: you assume that the left hand side (LHS) variable is being impacted by the right hand side (RHS) variables (and not the other way around). You are further assuming that you included everything that is correlated with LHS and RHS variables, so, no omitted variable bias. Finally, you are saying that each of your measurements are independent of each other.

Control Variables Under these assumptions, some of your LHS variables will be "controls" - you included them only to avoid the omitted variable bias. Others are for hypothesis testing, you can call them "variables of interest". You want to interpret the p-values of the variables of interest. If the p-value is high, than this means that the impact is not large enough to be distinguishable from statistical noise. In formal terms, you fail to reject the hypothesis that the RHS in question impacts the LHS variable.

Should they be included in the model?

Undergrad-level response: If you have reason to believe that they have an impact on the RHS variable and the LHS variable, then yes, include them to avoid omitted variable bias. If you are confident that they are not related, excluded them, because they will unnecessarily increase the variance estimate, increasing the likelihood of falsely failing to reject the null hypothesis - this is also known as a type II error, you would fail to estimate an impact, even though there is one.

Grad-level response: many consider the omitted variable bias to be a serious problem in academia. Many academics may want to see additional control on a model. Consider reading the section "good cop, bad cop" in Mostly Harmless Econometrics - it's somewhere in chapter 1 or 2.

• Because correlation is not causation, it is invalid to claim that linear regression models mean you "assume [$y$] is being impacted by" the regressors. Moreover, there is not even an implicit assumption that the model includes "everything that is correlated" with all variables: if that were the case, we could never get started!
– whuber
Commented Apr 25 at 12:10
• I agree with @whuber, and I apologize for any confusion. What I meant was that when we place the a variable on the left hand side and call it y, and place others on the right hand side and call them x or x_1, we are assuming that x causes y, we are not proving it. It is important to make such underlying assumptions explicit. Otherwise, we can only say that a LHS variable is associated with the the RHS variable of interest, after the controlling for the other RHS variables. Commented Apr 28 at 18:57
• I'm sorry, but that's not remotely correct: regression models make no assumption about causation. That is a fundamental distinction.
– whuber
Commented Apr 29 at 13:37
• No need to be sorry! Modellers put the variable that they assume to be dependent on the LHS, hence the name "dependent" variable. Hence, the modeller makes the implicit assumption that the LHS is causally dependent on the RHS variables. Commented May 1 at 1:50
• That just is not correct, -1.
– whuber
Commented May 1 at 2:03