1
$\begingroup$

I just heard a PhD student claim:

The $t$-test is really fundamental and nobody in academy is gonna do it.

He added that these tests serve more as a beginning exercise in order to get an idea of where to go from there.

EDIT: So - are t-tests rarely utilized in highly-ranked research papers and if so what are the potential reasons?

|cite|improve this question
$\endgroup$
  • 10
    $\begingroup$ I am voting to close this as off topic b/c it's opinion-based. The only rational answer I can think of is that this PhD student is wrong and doesn't know what he or she is talking about. $\endgroup$ – AdamO Jul 12 at 19:52
  • $\begingroup$ I just read the "rules" for asking questions - so my apologies for this one. But many thanks for answering it anyway! $\endgroup$ – Etienne Jul 12 at 19:57
  • 1
    $\begingroup$ @AdamO: I strongly disagree. Of course, we should not only say why it is wrong but also elaborate it, which may help the asker and others to further their understanding of how statistical tests work in general. $\endgroup$ – Wrzlprmft Jul 12 at 20:16
  • 2
    $\begingroup$ "Nobody is going to use a t test" is not opinion-based question - either people do use it, or not. Voted to reopen. (Answer: they do use it, often appropriately, often inappropriately. Usage may differ strongly between disciplines.) $\endgroup$ – S. Kolassa - Reinstate Monica Jul 12 at 20:35
  • 2
    $\begingroup$ The phd student might have meant "elementary" instead of "fundamental" and might have been referring to his/her supposition that a journal would not likely accept a submission that made heavy use of these tests. Even if that was false, but it was perceived to be true, then that could explain submission patterns. I do think the question is off topic, but because it's too broad, not because it's primarily opinion-based, but those two things usually go together. The statement could be accurate if it was clarified and qualified further. But right now there are just too many ways to answer this. $\endgroup$ – Taylor Jul 12 at 20:44
3
$\begingroup$

Different statistical tests answer different questions. The $t$ test comes in many flavours, but the most simple one answers the question whether the mean of a population is different from zero¹. The sign test answers the slightly different question whether the median of a population is different from zero. The Kolmogorov–Smirnov test answers the question whether two populations differ. Tests also differ in their requirements on the data, but this can be considered a detail of the question they answer. (The $t$ test also has its requirements that are however not relevant to this question.)

When teaching the concepts of hypothesis tests, I (and probably many others) focus on the $t$ test and sign test, because they answer the arguably most simple question a statistical test can answer. As the students usually have enough trouble wrapping their mind around the concept of statistical testing, I do not want them to additionally burden them with understanding a more complicated scenario, such as investigating correlations. On top, the mathematical backbone of these tests are easy to grasp for those audiences who can appreciate them.

However, the $t$ test answering a simple question does not make it a bad test that should never be used. If you have the corresponding question on your data, it may be exactly the right tool. Note that the fact the statistical question posed by your academic research is simple doesn’t mean that your entire research is simple. There is a lot of sophisticated academic research that correctly employs the $t$ test. Of course, researchers should make sure that they really need the answer that the $t$ test provides, in particular ensuring that its requirements are fulfilled, and sadly they often fail at this. But that’s a problem with every statistical test – it has nothing to do with the $t$ test answering a simple question.

So, the student’s statement is roughly as wrong as the following:

Addition is the most fundamental arithmetic operation and only a beginner’s exercise in mathematics. Nobody in academia uses it nowadays.

¹ or more precisely: If I sample from a population with zero mean and certain other properties, how likely is it that the mean of the samples is as extreme as that of the given sample?

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks a lot for the explanation - your way of simplifying the concepts really appeals to me. Assuming that this thread will be closed soon, may I ask you on a recommendation for a book on statistical testing (preferably a more application-oriented one)? I had courses in satistics and econometrics in university, but I still realize that in many cases the penny hasn't dropped yet. $\endgroup$ – Etienne Jul 12 at 22:24
  • $\begingroup$ @Etienne: Your question is already closed (meaning no new answers can be posted). It is not deleted and unless some high-reputation users vote for it to be, it won’t at its current stage. $\endgroup$ – Wrzlprmft Jul 13 at 8:07
  • $\begingroup$ @Wrzlprmft provided good answers, and I'll just elaborate that in many cases the t-test is naive because of its equal variance and normality assumptions. We should routinely relax those assumptions and use the Bayesian t-test as described here. We need to stop pretending that either the assumptions are true or that our checking of the assumptions was reliable in non-huge samples. $\endgroup$ – Frank Harrell Jul 13 at 11:04

Not the answer you're looking for? Browse other questions tagged or ask your own question.