There are several posts on this site talking about the need of normality when interpreting the meaning of the p.value of a linear regression. But not much I think is said about how to deal with non-normal data set. On this post, they give some solutions when the distribution is long-tailed.

I'm dealing with a case where my residuals (and my dependent variable) has a multimodal distribution (as it can be seen on the following Kernel density graphs) and takes discrete values (as it can be seen in the other graphs). My model takes "FP" as dependent variable and "Design Complexity" and "Sample size".

On the following graph, each point and whiskers represent 20 points. I strongly expected to have an effect of "Design Complexity" but didn't know wether "Sample size" may have an effect. As I couldn't reject the possibility that an interaction might exist between "Sample size" and "Design Complexity" I observed both.

enter image description here

Here is the R code for ths model.

summary(aov(FP~Obs.size*Design.Complexity, data=data.and.factors))

But my residuals are definitely not normally distributed:

note: All the following graphs are presented in both kernel density and regular x-y plot


enter image description here enter image description here

My dependent variable is not normally distributed either and its distribution change from one "Design Complexity" value to another (see below)

Distribution of FP

enter image description here enter image description here

Distributions of FP for Design Complexity equal 1, 2, 3, 4, 5 and 6. enter image description here enter image description here

How can I get trustfull p.values ?

  • 4
    $\begingroup$ I agree with @NickCox that this is not as bad as you think. You can try robust regression (rlm in the MASS package), or a permutation test (lmPerm package), if you're worried. Switching the roles of sample size and complexity in your first plot would probably give more informative results ... $\endgroup$
    – Ben Bolker
    Jun 3, 2013 at 0:57

1 Answer 1


I wouldn't call that multinomial. Residuals are measured on a continuous scale, but they have multimodal distribution, rather than multinomial.

By the way, a kernel density plot makes the modality more difficult to judge. Some kind of strip plot or strip chart would be helpful.

Commenting on your model would be easier with some scientific context, but sample size is less convincing as a control than design complexity. If sample size is an issue at all, would you expect an interaction? Would you expect a linear relationship?

Design complexity has a strong effect. If you start with the much simpler model in which design complexity is the only factor, then what is crucial is the distribution of residuals at each level of complexity.

My bottom line is that normality of residuals is less of a big deal than you seem to think. You seem to have approximate symmetry of residuals, and perhaps your P-values will be a little untrustworthy, but they are usually dubious any way.

  • $\begingroup$ could you justify why non normality of residuals is not a big deal? I have a very similar question to this open right now $\endgroup$
    – Trajan
    Aug 1, 2020 at 18:48
  • $\begingroup$ The essence is already in the last sentence: "perhaps your P-values will be a little untrustworthy, but they are usually dubious any way". You got good answers already at stats.stackexchange.com/questions/481090/… $\endgroup$
    – Nick Cox
    Aug 1, 2020 at 19:13
  • $\begingroup$ Are you saying that the p-values will be (tighter/less variance?) if you use normally distributed errors instead of uniformly distributed? $\endgroup$
    – Trajan
    Aug 1, 2020 at 20:38
  • $\begingroup$ I think you need a simulation tailored to your circumstances to get a good answer to that. $\endgroup$
    – Nick Cox
    Aug 2, 2020 at 9:06
  • $\begingroup$ why do you need a simulation? you already know the distribution properties of the residuals themselves $\endgroup$
    – Trajan
    Aug 2, 2020 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.