1
$\begingroup$

I am trying to guide students towards the correct statistical tests in biology. I found the decision tree attached at https://dzchilds.github.io/stats-for-bio/choosing-models-and-tests.html and have added some text in red as I am trying to understand if there are some errors / omissions in it - or if I am misunderstanding some of the tests.

I have 4 main questions questions (you can see in red text in the graphic the matching locations by number):

  1. The Wilcoxon (signed rank) test is listed in two places. Would it be correct to also say the Sign test could be applied in the situations given, but that the Wilcoxon (signed rank) test is preferred?

  2. The t-test is listed in three places. However, would the z-test not also be an option? My understanding is that if we are working with comparing ratios/proportions we should not use the t-test but use the z-test instead. Additionally, I see recommendations that if we have 30 or more samples we should use z-test instead of the t-test. If what I just said is correct, then this graphic should show z-test as an option at all 3 locations that t-test is shown? If I am not correct, how would z-tests be incorporated correctly into this decision tree?

  3. Near the bottom they list the Kruskal-Wallis test. I believe this chart neglects the Friedman test option, which could be added by adding another branching question about whether the samples are related (paired) or independent (unpaired). This would guide students to choose the Friedman test for related (paird) samples and the Kruskal-Wallis test for independent (unpaired) samples. If I am wrong about this, where would the Friedman test be shown in this chart?

  4. The chart does not include the F-test, which I think based on how it is organized would be better addressed by adding a 4th question about whether they are looking at a difference in variances or not. Does that make sense as the chart is organized, or would it fit within the existing questions somehow?

Many thanks...

enter image description here

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I do not see "Welch" mentioned either in terms of a $t$-test or ANOVA. So my immediate instinct is that this looks like a traditional teaching guide with assumptions rarely justified in practice, rather than a practical tool. Similarly Tukey rather than Games-Howell post hoc. Unless you have a theoretical belief in equal variances, just assume they could be different and use a suitable test rather than checking empirically $\endgroup$
    – Henry
    Commented Oct 15, 2021 at 17:02
  • 1
    $\begingroup$ For your z-test versus t-test concern, the t-test is indistinguishable from the z-test at large N. So for simplicity it's probably just as well to omit the z-test as an alternative. A more critical omission is how to handle binary or count outcomes (generalized linear models) or times to events (survival models), which are also important in biology. The danger with this type of chart is that students will try to force their data of those types into one of the listed tests even if the test they identify from the chart is inappropriate. $\endgroup$
    – EdM
    Commented Oct 16, 2021 at 16:17

1 Answer 1

Reset to default
4
$\begingroup$
  • You might add a post-hoc for Kruskal-Wallis. I recommend Dunn test (1964).
  • You might add some kind of nonparametric analog of two-way anova. Maybe Scheirer–Ray–Hare test, or Aligned Ranks Transformation anova.
  • Saying "data normally distributed" as a condition of two-way anova is really misleading for students. There is so much misinformation about the assumptions of general linear models, and it causes so many problems for students and young researchers. I try to address these misunderstandings continually on ResearchGate. Please don't be one of the people who's resources I say to "discard entirely" because they don't correctly address the assumptions for these models.
  • For anova-type tests, relative homogeneity of variance may be a bigger concern than the conditional normality of the data.
  • But more importantly for this project, in general I strongly discourage writing things like "data are normal" in a flow chart like this. Why? 1) No data are truly normal. Students should say this to themselves every morning. 2) These tests are somewhat robust to deviations from their model assumptions. 3) It's not really the distribution of the data that we are concerned with. Students should say this to themselves three times every morning.
  • Maybe just say something like "Parametric test is appropriate" on the flowchart, and then have a separate sheet that addresses what this means. Some of my recommendations: a) Start with the assumptions of a general linear model. For example, in Montgomery, Design and Analysis of Experiments. Or at the following presentation, see slide 8, and then slides 19 - 22. https://staff.emu.edu.tr/adhammackieh/Documents/courses/ieng581/lecture-notes/ch03.pdf . From there, you can describe the simplified assumptions for t tests, and variants for e.g. Welch's t test and anova. b) Don't use tests for normality or homogeneity; Use q-q plots or histogram of residuals; Use plots of residuals vs. predicted values. c) Consider that tests are somewhat robust. d) Consider where your data come from, and if an assumption of conditionally normal distribution makes sense.
Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ (+1) No data are truly normal. (or Poisson, gamma, etc...) Another way of saying this, and in my opinion more effective, is to consider that we fit distributions (models) to the data, not the other way round. The data is what it is, it doesn't follow any distribution, and we seek the model that gives the simplest explanation without excessive bending. My 2p - agree? $\endgroup$
    – dariober
    Commented Oct 16, 2021 at 17:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.