3
$\begingroup$

Background

As I understand it (as a novice), parametric tests don't use your actual data as input but rather parameters extracted from your data which describe a distribution. The tests use the distributions which are modelled on your data.

Many researchers who have a limited understanding of the technical details of statistics learn about the normal distribution and that's about it. From what I have read, there are numerous potential distributions which could describe a set of data and the techniques for fitting one of them to the data and evaluating the goodness of fit are likewise numerous. To be honest, it is all hopelessly over my head.

Question(s)

Given that the probability of me attaining a level of understanding sufficient to actually model and evaluate the fit of a distribution to my data lingering around $0$,

  1. how important is it to verify that a given distribution, such as the normal distribution, actually models my data?
  2. If it is very important and I have no idea how to do it, would it be better to just stick with non-parametric tests which seem to make no assumptions about the shape of the data?

Supplemental

Note that I do understand the importance and usefulness of distributions, I just don't understand how to ensure that a given distribution really represents my data, so I would be very interesting in defaulting to non-parametric tests as an alternative to avoid invalid conclusions due to inappropriately assuming a distribution.

Also, the fields I'm alluding to typically have $n < 30$ or so.

$\endgroup$

1 Answer 1

3
$\begingroup$

As I understand it (as a novice), parametric tests don't use your actual data as input but rather parameters extracted from your data which describe a distribution. The tests use the distributions which are modelled on your data.

They do use your actual data. Some parametric tests are able to take advantage of statistics that summarize all the information down to a single statistic (or a few). For example, with normal data, all the information is contained in two sufficient statistics (equivalent to knowing the sample mean and variance).

Some other parametric tests cannot reduce the data in that fashion; the likelihood unavoidably resists reduction to some smaller set of values.

Many researchers who have a limited understanding of the technical details of statistics learn about the normal distribution and that's about it.

True - but you should be equally wary of trying to overgeneralize from what you understand from that.

From what I have read, there are numerous potential distributions

To say the least. The actual number of potential distributions is uncountably infinite. Even recording basic details of the distributions people have named and used would be large enough to fill many volumes.

how important is it to verify that a given distribution, such as the normal distribution, actually models my data?

It depends on what you're doing. Some procedures are very sensitive to normality, some are not. The sensitivity can also depend on things like sample size, for example. With t-tests, ANOVA and regression, large samples help.

If it is very important and I have no idea how to do it, would it be better to just stick with non-parametric tests which seem to make no assumptions about the shape of the data?

This is not entirely accurate. They don't make parametric assumptions, but they do make assumptions. For example, the Wilcoxon signed rank test assumes symmetry of the distribution of pair-differences; that's certainly an assumption about the shape.

Note that I do understand the importance and usefulness of distributions, I just don't understand how to ensure that a given distribution really represents my data

Ensure? Nobody can do that.

You can check the reasonableness of an assumption, of course, via diagnostics (e.g. a Q-Q plot), but more critical is having an understanding of the likely impact on your inference. There are a number of ways of building such an understanding.

so I would be very interesting in defaulting to non-parametric tests as an alternative to avoid invalid conclusions due to inappropriately assuming a distribution.

Where feasible, nonparametric and robust procedures can be very useful; I recommend learning about permutation tests and bootstrapping intervals, both of which can be handy as long as sample sizes are not too small.

But consider a two-sample t-test for example; unless the sample size is quite small or the non-normality is strong, it's often more affected by the assumption of equality of variance than the assumption of normality -- and a nonparametric procedure used to test for equality of means can be as badly affected by the violation of that equal-variance assumption as the t-test.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.