I'm trying to determine whether I should be using an ANOVA or a non-parametric test. My DV is rating of confidence on a Likert scale (1-5), and 4 categorical IVs (n = 496).

Data distribution within each IV was non-normal, all Shapiro-Wilk tests p < .001, and Levene's test was significant for 2 of the 4 IVs. Also, there are 13 outliers (which were scores of 5 on the Likert scale). After excluding outlier cases, only 1 IV had a significant Levene's test - IV participant sex, in which the groups sizes were very unequal (70% female).

Conducting an ANOVA with and without outliers finds similar results, equality of variance, no interactions, 1 significant factor (with outliers), 2 significant factors (without outliers). Using a kruskal-wallis test also produces the same results as the ANOVA.

I'm not sure of the most appropriate analysis to use, parametric or non-parametric? Have I violated too many assumptions for the ANOVA?

Any advice is much appreciated, thanks.

  • $\begingroup$ Why would you test the distribution of IVs? When you say there were outliers, how did you decide that, on which variables were they found, and why would you exclude them? Why do you think any regression assumptions are violated? $\endgroup$ – Glen_b Oct 6 '13 at 7:30
  • $\begingroup$ I checked the distribution of the DV within each IV. Outliers were mild, within all IVs, and legitimate values, so probably shouldn't be excluded. My supervisor advised me to see whether there's much of a difference between an ANOVA with and without outliers. I'm unsure of his direction regarding my analyses... And I thought for a valid ANOVA I needed no outliers, normal distribution of data, and homogeneity of variance. Is this not correct? $\endgroup$ – Lyss Oct 6 '13 at 8:47
  • $\begingroup$ It's not correct - at least not in respect of IVs. There's no assumption whatever about the distribution of IVs. (Checking whether very discrepant values of IVs make much a difference to the analysis is reasonable, however). You don't have normal distribution of data in any case, you can be certain of that without even looking at any data; it's pointless to imagine that normality is even likely, since it simply won't be the case. ... $\endgroup$ – Glen_b Oct 6 '13 at 9:13
  • $\begingroup$ (ctd)... If you're doing hypothesis testing or confidence intervals you will want the conditional distribution of your data to be not far from normal (which you normally check via residuals), and you'll want your conditional variances not to vary heavily (but this is something you check conditional on some model). When you say "outliers were mild within IV's" do you mean you checked the distribution of your DV within values of each IV considered one at a time (i.e. ignoring all other IVs)? $\endgroup$ – Glen_b Oct 6 '13 at 9:16
  • 1
    $\begingroup$ While that would very often find serious problems, the actual assumption is that the distribution is normal with constant variance within each combinations of IVs ... which is why people tend to examine residuals to check the assumptions. In fact, it's possible for the assumptions to all be fine (i.e. reasonably close to true, though not actually true), but just checking conditional on each IV one at a time to look quite poor for at least some IVs. $\endgroup$ – Glen_b Oct 6 '13 at 10:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.