There are several posts on this site talking about the need of normality when interpreting the meaning of the p.value of a linear regression. But not much I think is said about how to deal with non-normal data set. On this post, they give some solutions when the distribution is long-tailed.
I'm dealing with a case where my residuals (and my dependent variable) has a multimodal distribution (as it can be seen on the following Kernel density graphs) and takes discrete values (as it can be seen in the other graphs). My model takes "FP" as dependent variable and "Design Complexity" and "Sample size".
On the following graph, each point and whiskers represent 20 points. I strongly expected to have an effect of "Design Complexity" but didn't know wether "Sample size" may have an effect. As I couldn't reject the possibility that an interaction might exist between "Sample size" and "Design Complexity" I observed both.
Here is the R code for ths model.
But my residuals are definitely not normally distributed:
note: All the following graphs are presented in both kernel density and regular x-y plot
My dependent variable is not normally distributed either and its distribution change from one "Design Complexity" value to another (see below)
Distribution of FP
Distributions of FP for Design Complexity equal 1, 2, 3, 4, 5 and 6.
How can I get trustfull p.values ?