1
$\begingroup$

I am using regression with planned contrasts and would like to test statistical assumptions. Assumptions are normally tested on the residuals of the regression model, but in this case, I don't know if it makes sense because the predictor variable is categorical (i.e., group) and contrasts are only tested later (one contrast at a time, meaning two groups at a time).

For example, using the amazing performance package (which needs package see for the plot), we can see for the normality, homoscedasticity and homogeneity of variance plots, the observations cluster in three points only:

library(performance)
library(see)
mod <- lm(mpg ~ factor(cyl), data=mtcars)
check_model(mod)

enter image description here

While normally the observations would be distributed more equally like this:

mod2 <- lm(mpg ~ disp, data=mtcars)
check_model(mod2)

enter image description here

Question

Which option below is the correct/best one, given the situation? (Feel free to suggest another one) (Edit: Also note that I am using the assumption of normality for the sake of simplicity and conciseness but I am interested in all assumptions for this situation)

(a) Assess normality of the dependent variable

shapiro.test(mtcars$mpg)

    Shapiro-Wilk normality test

data:  mtcars$mpg
W = 0.94756, p-value = 0.1229

(b) Assess normality of the residuals of the whole model

shapiro.test(mod$residuals)

    Shapiro-Wilk normality test

data:  mod$residuals
W = 0.97065, p-value = 0.5177

(c) Assess normality of model residuals for each group contrast (combination of two groups) by excluding the third group manually before respecifying the regression

mod1 <- lm(mpg ~ factor(cyl), data=mtcars[which(mtcars$cyl!=4),])
mod2 <- lm(mpg ~ factor(cyl), data=mtcars[which(mtcars$cyl!=6),])
mod3 <- lm(mpg ~ factor(cyl), data=mtcars[which(mtcars$cyl!=8),])

shapiro.test(mod1$residuals)

    Shapiro-Wilk normality test

data:  mod1$residuals
W = 0.9515, p-value = 0.3636

shapiro.test(mod2$residuals)

    Shapiro-Wilk normality test

data:  mod2$residuals
W = 0.95956, p-value = 0.4058

shapiro.test(mod3$residuals)

    Shapiro-Wilk normality test

data:  mod3$residuals
W = 0.96698, p-value = 0.7393

Note: With option (c), I believe the assumptions would not apply to the model as a whole, but to each comparison test separately. So say you have 2 models with 3 contrasts each, instead of having assumption checks for the two models, you would have assumptions checks for the 6 contrasts.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The distributional assumptions are the same with categorical predictors as with numerical predictors; namely, the conditional distributions are all assumed to be normal with common variance. However, with categorical predictors you do not have to resort to looking at residuals, because you can investigate the conditional distributions directly by subsetting the data for the different levels of the categorical predictor.

In the case of numerical predictors, there is typically little data within the subsets to allow meaningful investigation of the conditional distributions. That is why people often resort to looking at the residuals in such cases.

So your "method 3" looks ok, except (i) there is no need to look at residuals, just look at the raw data, and (ii) use the S-W test only as an afterthought. Instead, look mainly at the histograms and q-q plots, and use your subject matter knowledge.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Oh, that is such useful information, thank you! So basically, something like this for each variable? mod1 <- mtcars[which(mtcars$cyl!=4),"mpg"] shapiro.test(mod1); mod2 <- mtcars[which(mtcars$cyl!=6),"mpg"] shapiro.test(mod2); mod3 <- mtcars[which(mtcars$cyl!=8),"mpg"] shapiro.test(mod3); But would you say that looking at the dependent variables without subsetting for groups is incorrect? (This is what we had done before) In the sense that the correct approach is to check assumptions for each contrast and not each model, correct? $\endgroup$
    – rempsyc
    Dec 24, 2020 at 2:04
  • 2
    $\begingroup$ Right, the assumptions of regression all concern the conditional distributions of $Y$. Empirically, these refer to subsets of data based on values of $X$. The marginal distribution of $Y$ can be useful to examine just for understanding, but there is no regression assumption about it. $\endgroup$
    – BigBendRegion
    Dec 24, 2020 at 13:22
  • $\begingroup$ Oh thank you for this comment, it really helped me understand. The key is indeed the conditional distributions of $Y$, such that the "condition" is normally a continuous $X$ (thus the residuals for a lack of better option), but in this case the condition is a categorical $X$ (this is the better option and it is available) so there are only three "points". [On a side note, agreed that visual assessment of diagnostic plots should be used generally. I used Shapiro here for conciseness and simplicity of the demonstration]. $\endgroup$
    – rempsyc
    Dec 24, 2020 at 15:11
  • $\begingroup$ Another question: Assessing normality of the raw data is easy enough. However, I assume that the "no autocorrelation of residuals" assumption does not apply since I am not looking at residuals anymore? Or is there a variant I should use here? What about homoscedasticity and homogeneity of variance? Do they apply or not in this case? $\endgroup$
    – rempsyc
    Dec 24, 2020 at 21:21
  • 1
    $\begingroup$ All the same assumptions in either case. The residual uncorrelatedness assumption can as easily be framed as conditionally (given $X$) uncorrelated $Y$ values. With categorical $X$ you can again evaluate the assumption using the raw $Y$ values; with numerical $X$ you again need to use residuals because there is little data in the subsets. $\endgroup$
    – BigBendRegion
    Dec 24, 2020 at 21:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.