4
$\begingroup$

enter image description here

Here is a regression of a continuous response variable on a factor (religion) with 8 levels in R. I am wondering if any conclusions could be drawn from those p-values that are greater than $0.05$?

Aside I remember that the p-values display in summary() for MLR are the t-test values, it only provides over and extra information when other variables are included. Even some variables have p-values that greater than $0.05$, it doesn't mean it's not useful in the model, to be more accurate, we need to have a sequential F-test to check the usefulness.

Q1 what about the p-values for different levels (not variables)? Can we conclude any levels is not important based on summary(), if not, what kind of test I need to have to say a specific level is not important, rather than this categorical variable?

Q2 If p-values $>0.05$ in summary() is not sufficient to say it's not an important variable, what about the p-values $<0.05$ in summary(), is this situation sufficient to say it's an important variable?

Q3 If the p-values in summary() cannot tell much in the MLR, why do we run summary() for the regression nearly every single time? What we can know from MLR summary() p-values?

$\endgroup$

1 Answer 1

1
$\begingroup$

I am wondering if any information I could draw from those p-values that are greater than $0.05$?

Short answer: Just that they do not differ significantly from the intercept at $\alpha = 0.05$. Whether this information is useful depends on your research question.

F-test

The omnibus test you are describing is actually included in the output. It is the last p-value in the bottom right. If all group means were equal, there would be a $0.009\%$ of observing at least this large a difference.

Interpreting variable categories in the output (Q1)

If you run something like levels(survey$religion), you'll notice that one category is missing from the output. That category is placed in the intercept. After all, if you don't belong to any of the other groups, you must be the remaining group.

All the estimates and $p$-values of the other categories are a comparison to the reference group, or in other words, the group in the intercept. The estimate for Jewish isn't $2.1870$, it's $7.5361 + 2.1870$. Muslim differs significantly from the reference group. Etcetera.

If a category has a high $p$-value, that simply means it does not differ significantly from the reference group. The variable religion is added to your model, and if any of its categories differ sufficiently, then religion has a significant effect. If this is your research question, then all you need is the omnibus $F$-test.

Taking out categories (Q2)

You can't take out certain categories and leave others in. What you could do to achieve the same effect though, is to merge similar categories. Then again, this isn't a decision that you should make based on the $p$-values in the output, but rather at the start of the analysis (for example because two groups are theoretically similar and both quite small).

What is the point of the summary (Q3)

No one is forcing you to use summary. If all you want is the estimates, then coef(model) works fine too. But the summary includes other relevant information: Jewish has a large difference from the intercept, but then again, its standard error is also large. The $p$-value just gives you a simple guideline for the size of the estimate compared to its standard error.

Yes, religion has a significant effect, but a model with just this variable only explains a miniscule $2.6\%$ of the variance in the outcome variable (Multiple R-squared). Just looking at the coefficients, or their $p$-values alone would not have given you this information.

"Drinks" sounds like a count (see other comments) and is probably strongly right-skewed. You can see this by comparing the extrema and quartiles shown in the residuals tab.

Other comments

  • If "drinks" is a count, you shouldn't be using normal linear regression, but instead a model for count data, like a Poisson or negative binomial GLM (glm(..., family = "poisson") or glm.nb(...), respectively).
  • If you are interested in comparing other groups, you can change the reference group using relevel.
  • If you are interested in more than one comparison, you should adjust for multiple testing.
$\endgroup$
4
  • $\begingroup$ Thank you so much for your reply! Extremely useful!!!! I still got a question about the example you provided The estimate for Jewish isn't $2.1870$, it's $7.5361+2.1870$ As this $p$-value $>0.05$, and we have no enough evidence to reject null, which it has the same value as the intercept, then why do we still take $2.1870$ into account for Jewish? $\endgroup$
    – LJNG
    May 24, 2021 at 21:33
  • $\begingroup$ Because that is still the point estimate. The fact that it does not differ significantly from the intercept is not always relevant. If you want to show that they don't actually differ significantly, you could add error bars in a figure. $\endgroup$ May 25, 2021 at 7:48
  • $\begingroup$ Thank you for your reply. I still do not get your point in your reply. But if the following explanation convincible: no enough evidence to reject they are the same, does not imply they are the same. This is how I convince myself $\endgroup$
    – LJNG
    May 26, 2021 at 8:53
  • 1
    $\begingroup$ Yes that is a valid interpretation. It is also often stated as: Absence of evidence is not evidence of absence. Or in the case of hypothesis testing: Failure to reject the null-hypothesis is not evidence for the null-hypothesis. $\endgroup$ May 26, 2021 at 9:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.