Normality and homoscedasticity are lacking: Is transformation necessary?

Question

I'm a student and I'm very new at this so I wanted to ask what to do. I have a data set and one of the groups didn't pass Shapiro-Wilk normality test (p value = 0.01) but testing with model residuals everything looks alright (p value = 0.49). Another problem is with homoscedasticity because it barely passes Levene's test (p value = 0.047) and studentized Breusch-Pagan test (p value = 0.089). My supervisor told me I'll need to normalize and transform the variable. I can't use log transformation because some of the values are 0 (and it gives me -Inf which I cannot use in further tests) so I did square root transformation. While the p values in homoscedasticity tests improved, the p values normality tests became worse. So my question was do I need a different transformation method? If so, which one? But after reading some more I found out that "...your data don't have to be perfectly normal and homoscedastic; parametric tests aren't extremely sensitive to deviations from their assumptions." (McDonald, J.H. 2014. Handbook of Biological Statistics (3rd ed.)). So, are the transformations really necesary in this case?

Everything I've done in detail (and in R code):

mydata <- tibble(group, y) %>%
  dplyr::mutate(group = factor(group, ordered = FALSE))

mydata %>% 
  group_by(group) %>%
  dplyr::summarize(
    p_value = shapiro.test(y)$p.value,
statistic = shapiro.test(y)$statistic
    )

model_anova <- lm(y ~ group, data = mydata)
residuals    <- residuals(model_anova)
ggdensity(residuals, fill = "grey", rug = TRUE)

shapiro_test(residuals)

rstatix::levene_test(y ~ group, data = mydata)

lmtest::bptest(model_anova)

Answer 1

Thanks for showing the data, but until the whole world uses R, your use of R syntax will be at best awkward and at worst not as helpful as you hope for those who don't use R. So for those people here is another listing of the your data.

y  group
1.5 "C"
  0 "C"
  1 "C"
  1 "C"
  1 "C"
  1 "C"
  1 "C"
  1 "C"
 .5 "C"
1.5 "C"
 .5 "C"
  1 "C"
 .5 "C"
  1 "C"
  1 "C"
 .5 "C"
  1 "C"
  2 "C"
 .5 "C"
2.5 "C"
1.5 "C"
  0 "C"
  1 "C"
  0 "C"
  2 "C"
 .5 "C"
 .5 "C"
1.5 "C"
  1 "C"
  2 "C"
 .5 "P"
  1 "P"
  2 "P"
2.5 "P"
1.5 "P"
  0 "P"
  2 "P"
  1 "P"
1.5 "P"
2.5 "P"
1.5 "P"
 .5 "P"
  1 "P"
3.5 "P"
  2 "P"
2.5 "P"
  3 "P"
  1 "P"
  3 "P"
3.5 "P"
1.5 "P"
  4 "P"
1.5 "P"
2.5 "P"
3.5 "P"
  1 "P"
  2 "P"
  2 "P"
  2 "P"
  2 "P"
2.5 "P"
  1 "P"
2.5 "P"
  1 "P"
  2 "P"
 .5 "P"
  2 "P"
2.5 "P"
  0 "P"
 .5 "P"
1.5 "P"
 .5 "P"
1.5 "P"
  1 "P"
 .5 "P"

and indeed that may not be convenient for everyone.

This normal quantile plot tells almost the whole story that we can discern.

Sure, there is some slight skewness and heteroscedasticity visible there to the experienced eye, but neither is a big deal. And any idea that normal distributions are the goal here is compromised by the rounding of values as multiples of 0.5. Other way round, that requires flagging and some discussion. More: it seems a fair guess that your outcome cannot be negative and even that there may be an upper limit too: such a bound or bounds is also problematic for the ideal of a normal distribution.

In practice, a $t$ test whether using equal or unequal variances or a regression that ignores or respects heteroscedasticity all give unequivocal results, a convincing difference between the groups on this outcome. I show Stata results below and am confident that R code exists for the equivalent.

I disagree with your supervisor here. There is some departure from ideal assumptions but it is at worst a distraction. There is no need for, and no likely gain from, a transformation. I write as someone highly positive about using transformations -- when they really are needed and helpful.

. moments y , by(group)

----------------------------------------------------------------------
    Group |          n        mean          SD    skewness    kurtosis
----------+-----------------------------------------------------------
        C |         30       1.000       0.616       0.450       2.913
        P |         45       1.722       0.980       0.295       2.467
----------------------------------------------------------------------

. encode g, gen(Group)

. regress y i.Group

      Source |       SS           df       MS      Number of obs   =        75
-------------+----------------------------------   F(1, 73)        =     12.86
       Model |  9.38888889         1  9.38888889   Prob > F        =    0.0006
    Residual |  53.2777778        73  .729832572   R-squared       =    0.1498
-------------+----------------------------------   Adj R-squared   =    0.1382
       Total |  62.6666667        74  .846846847   Root MSE        =     .8543

------------------------------------------------------------------------------
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       Group |
          P  |   .7222222    .201361     3.59   0.001     .3209103    1.123534
       _cons |          1   .1559736     6.41   0.000     .6891451    1.310855
------------------------------------------------------------------------------

. regress y i.Group, robust

Linear regression                               Number of obs     =         75
                                                F(1, 73)          =      15.34
                                                Prob > F          =     0.0002
                                                R-squared         =     0.1498
                                                Root MSE          =      .8543

------------------------------------------------------------------------------
             |               Robust
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       Group |
          P  |   .7222222   .1844098     3.92   0.000     .3546939    1.089751
       _cons |          1   .1120584     8.92   0.000      .776668    1.223332
------------------------------------------------------------------------------

. ttest y, by(group)

Two-sample t test with equal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
       C |      30           1    .1124441    .6158818     .770026    1.229974
       P |      45    1.722222    .1461246    .9802339    1.427727    2.016717
---------+--------------------------------------------------------------------
combined |      75    1.433333    .1062605    .9202428    1.221605    1.645062
---------+--------------------------------------------------------------------
    diff |           -.7222222     .201361               -1.123534   -.3209103
------------------------------------------------------------------------------
    diff = mean(C) - mean(P)                                      t =  -3.5867
Ho: diff = 0                                     degrees of freedom =       73

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 0.0003         Pr(|T| > |t|) = 0.0006          Pr(T > t) = 0.9997

. ttest y, by(group) unequal

Two-sample t test with unequal variances
------------------------------------------------------------------------------
   Group |     Obs        Mean    Std. Err.   Std. Dev.   [95% Conf. Interval]
---------+--------------------------------------------------------------------
       C |      30           1    .1124441    .6158818     .770026    1.229974
       P |      45    1.722222    .1461246    .9802339    1.427727    2.016717
---------+--------------------------------------------------------------------
combined |      75    1.433333    .1062605    .9202428    1.221605    1.645062
---------+--------------------------------------------------------------------
    diff |           -.7222222    .1843803               -1.089708   -.3547362
------------------------------------------------------------------------------
    diff = mean(C) - mean(P)                                      t =  -3.9170
Ho: diff = 0                     Satterthwaite's degrees of freedom =  72.8047

    Ha: diff < 0                 Ha: diff != 0                 Ha: diff > 0
 Pr(T < t) = 0.0001         Pr(|T| > |t|) = 0.0002          Pr(T > t) = 0.9999

Cautions and extra comments: I've noticed that some economists, and some other groups, are extraordinarily diligent at following a ritual that every ideal in sight is tested for formally. But for example Shapiro-Wilk here just shows that your sample size is big enough for departure from exact normality to be detectable -- and part of that departure is the discreteness (and boundedness?) that makes the assumption or ideal condition of normality a dubious goal in any case. A normal quantile plot is always a good idea too. The point of a normal quantile plot is not so much that normality is really important but rather that you are showing distributions in a standardised way and other kinds of behaviour will be evident too.

I would rather see a normal quantile plot of residuals if I have to choose just one graph to check on their distribution.

The advice you quote from McDonald is perhaps a little more optimistic than is justified, but I think it's more nearly pointing in the right direction.

EDIT: Much of that needs rewriting. The problem morphed in comments into a quite different one: comparing binomial distributions with small numerators and denominator 2000.