0
$\begingroup$

First of all, sorry for catchy title, my question is not that broad as it suggests.

I just came to conclusion that I don't need parametric tests. Instead, I need some feedback if my reasoning makes sense. Here it is:

To use some parametric test (like t-test or ANOVA), we have to check assumptions of it (normality, heterogeneity of vaiance). But all the tests for assumptions (Shapiro-Wilk, Kolmogorov-Smirnov, Anderson-Darling, Bartlett, Snedecor's F and so on) test null hypothesis of assumption being met against alternative that it is violated.

So I can only reject null (conclude that assumption is violated) or say that I don't have enough evidence to do that. Typically, in later situation, I use parametric tests. This means that I use it when I do not actually know if assumptions are met.

My conclusion: I should use nonparametric test anyway.

My conclusion 2: I can use parametric test only when I have some "external" knowlegde about assumptions being met (like I somehow know exact distributions of my data).

That's all. I'll be greatful for any suggestions if above makes sense.

Edit: I know that parametric tests are more powerful when their assumptions are met. My concern is that we can never know that this is the case (unless we have "external" knowlegde). So, I think, using parametric tests is pointless since we don't know if we have data appropriate for them.

Analogy that comes to my mind: Imagine two stain removers: Parametric Remover is perfect for cherry juice stains and Nonparametric Remover is OK, but not perferct, for all the stains. Since I can never confirm that my stain is cherry juice, I should always use Nonparameric Remover.

Improve this question
$\endgroup$
8
  • $\begingroup$ Note that normality of residuals (which I assume you are alluding to testing) is useful, but not necessary. What is important is normality of parameter estimates. Which is asymptotically true even with non-normal residuals for already quite modest sample sizes. Also note that Kolmogorov-Smirnov is almost certainly an inappropriate test, since it can only be used for fully pre-specified distributions, not, e.g., if you estimate the residual variance from the residuals themselves. $\endgroup$
    – Stephan Kolassa
    Sep 17, 2019 at 9:28
  • $\begingroup$ I added some explanations above. $\endgroup$
    – Łukasz Deryło
    Sep 17, 2019 at 10:42
  • $\begingroup$ Your edit seems to claim: "unless we know assumptions are perfectly met, we should use the weaker/inferior alternative". But many tests/procedures are robust to small deviations from the assumptions, so this is not really true. And I also disagree with your claim that we can never know if those assumptions are met. We almost always have some external (or prior) knowledge, and some of those assumptions can be checked after fitting the model. $\endgroup$
    – mkt
    Sep 17, 2019 at 10:52
  • 1
    $\begingroup$ As @mkt writes: many tests are surprisingly robust against small deviations from assumptions. (And are only asymptotically valid, too.) Nonparametric tests typically come with lower power. There is always a tradeoff. Per my answer in the proposed duplicate, there was a time when I coded up complicated permutation tests routinely, because residuals were not normally distributed. (As I point out above, this argument is not valid, but tell that to a psychology reviewer.) Results were always extremely close to plain vanilla ANOVA. $\endgroup$
    – Stephan Kolassa
    Sep 17, 2019 at 11:29
  • 1
    $\begingroup$ One thought I didn't see addressed too directly: I wish we would teach less the idea of e.g. "Use a t-test unless the assumptions are violated, in which case use Mann-Whitney", and instead teach that different tests address different hypotheses. Mann-Whitney detects cases where an observation in one group is likely to be larger than another group, whereas t-test detects differences in means. In a given case, we might be interested in one of these hypotheses and not the other. Or we might want to test for a difference in medians, or of the 75th percentile, or the complete distribution. $\endgroup$
    – Sal Mangiafico
    Sep 17, 2019 at 13:12

0

Reset to default