59
$\begingroup$

Can someone explain to me why would anyone choose a parametric over a nonparametric statistical method for hypothesis testing or regression analysis?

In my mind, it's like going for rafting and choosing a non-water resistant watch, because you may not get it wet. Why not use the tool that works on every occasion?

$\endgroup$
  • 21
    $\begingroup$ It is to introduce inductive bias into the process of statistical inference. That is a fancy way of saying: if you know something, provide it as a hint. This hint may take forms such as providing the functional form of likelihood or prior distribution on the parameters. If your hint is good the outcome is better than without a hint, and if it is bad the result is worse. $\endgroup$ – Cowboy Trader Jul 30 '15 at 11:59
  • 6
    $\begingroup$ Not necessarily. Basically you are searching for something on a very large dessert. If somebody tells you that what you are looking for is confined in a certain territory then your chances of finding it improves. But if they mislead you with wrong information then no matter how much you search that territory, you will not find it. $\endgroup$ – Cowboy Trader Jul 30 '15 at 12:15
  • 17
    $\begingroup$ As a woodworker, I love the analogy at the end. Houses and furniture that are well-built and last are constructed using specialized tools. General-purpose tools are great for homeowners and for professionals who need to get a job done quickly or when using a crude or inappropriate tool won't make a quality difference that anyone cares about. Craftsmen, though, achieve the finest results by employing the right tool for the job, and indeed some things simply cannot be done without it. Nobody ever made a good dovetail with a general-purpose handsaw, for instance. $\endgroup$ – whuber Jul 30 '15 at 13:59
  • 3
    $\begingroup$ Whether you dig a ditch with a spade or a teaspoon you end up with a ditch. It's just that if you used the teaspoon, you're also older. $\endgroup$ – conjugateprior Jul 30 '15 at 14:53
  • 5
    $\begingroup$ Related (although perhaps not obviously so): If mean is so sensitive, why use it in the first place? $\endgroup$ – gung Jul 30 '15 at 21:12

14 Answers 14

23
$\begingroup$

Rarely if ever a parametric test and a non-parametric test actually have the same null. The parametric $t$-test is testing the mean of the distribution, assuming the first two moments exist. The Wilcoxon rank sum test does not assume any moments, and tests equality of distributions instead. Its implied parameter is a weird functional of distributions, the probability that the observation from one sample is lower than the observation from the other. You can sort of talk about comparisons between the two tests under the completely specified null of identical distributions... but you have to recognize that the two tests are testing different hypotheses.

The information that parametric tests bring in along with their assumption helps improving the power of the tests. Of course that information better be right, but there are few if any domains of human knowledge these days where such preliminary information does not exist. An interesting exception that explicitly says "I don't want to assume anything" is the courtroom where non-parametric methods continue to be widely popular -- and it makes perfect sense for the application. There's probably a good reason, pun intended, that Phillip Good authored good books on both non-parametric statistics and courtroom statistics.

There are also testing situations where you don't have access to the microdata necessary for the nonparametric test. Suppose you were asked to compare two groups of people to gauge whether one is more obese than the other. In an ideal world, you will have height and weight measurements for everybody, and you could form a permutation test stratifying by height. In a less than ideal (i.e., real) world, you may only have the mean height and mean weight in each group (or may be some ranges or variances of these characteristics on top of the sample means). Your best bet is then to compute the mean BMI for each group and compare them if you only have the means; or assume a bivariate normal for height and weight if you have means and variances (you'd probably have to take a correlation from some external data if it did not come with your samples), form some sort of regression lines of weight on height within each group, and check whether one line is above the other.

$\endgroup$
  • 3
    $\begingroup$ I understand that in the case of testing you don't have the same null, though I'm not sure if it makes any sense to say that a null is better than another null. But what about the case of prediction? Completely different story, still parametric vs. nonparametric dilemma. $\endgroup$ – en1 Jul 31 '15 at 14:35
20
$\begingroup$

As others have written: if the preconditions are met, your parametric test will be more powerful than the nonparametric one.

In your watch analogy, the non-water-resistant one would be far more accurate unless it got wet. For instance, your water-resistant watch might be off by one hour either way, whereas the non-water-resistant one would be accurate... and you need to catch a bus after your rafting trip. In such a case it might make sense to take the non-water-resistant watch along with you and make sure it doesn't get wet.


Bonus point: nonparametric methods are not always easy. Yes, a permutation test alternative to a t test is simple. But a nonparametric alternative to a mixed linear model with multiple two-way interactions and nested random effects is quite a bit harder to set up than a simple call to nlme(). I have done so, using permutation tests, and in my experience, the p values of parametric and permutation tests have always been pretty close together, even if residuals from the parametric model were quite non-normal. Parametric tests are often surprisingly resilient against departures from their preconditions.

$\endgroup$
  • $\begingroup$ It seems like the prevalent view that parametric methods are more powerful when their assumptions are met. But if that's the case then how come we have p-values to evaluate results from both approaches? I mean if a parametric test rejects a null hypothesis with a probability of at least 99%, how is that better from a nonparametric test rejecting a null hypothesis with a probability of 99%? Is the probability of 0.99 different in each case? That wouldn't make sense. $\endgroup$ – en1 Jul 30 '15 at 14:33
  • $\begingroup$ The null hypotheses are different between a parametric test and its nonparametric counterpart. Specifically, the null hypothesis for a parametric test contains a specific parametric assumption on the distribution of the test statistic (which will usually also be calculated in different ways for the two tests) - that's why it's called "parametric", after all! So the two p values have the same name, but are calculated based on different test statistics, which have different distributions under different null hypotheses. $\endgroup$ – Stephan Kolassa Jul 30 '15 at 15:14
  • $\begingroup$ ... and @StasK expressed it all much better than I did. $\endgroup$ – Stephan Kolassa Jul 30 '15 at 15:34
  • 2
    $\begingroup$ @StephanKolassa, I actually started writing a comment to your answer, and got carried away :) $\endgroup$ – StasK Jul 31 '15 at 13:56
11
$\begingroup$

While I agree that in many cases, non-parametric techniques are favourable, there are also situations in which parametric methods are more useful.

Let's focus on the "two-sample t-test versus Wilcoxon's rank sum test" discussion (otherwise we have to write a whole book).

  1. With tiny group sizes of 2-3, only the t-test can theoretically achieve p values under 5%. In biology and chemistry, group sizes like this are not uncommon. Of course it is delicate to use a t-test in such setting. But maybe it is better than nothing. (This point is linked to the issue that in perfect circumstances, t-test has more power than the Wilcoxon test).
  2. With huge group sizes, also a t-test can be viewed as being non-parametric thanks to the Central Limit Theorem.
  3. The results of the t-test are in line with the Student confidence interval for the mean difference.
  4. If variances heavily vary across groups, then Welch's version of the t-test tries to take this into account, while Wilcoxon's rank sum test can fail badly if means are to be compared (e.g. error probability of the first kind much different from the nominal level).
$\endgroup$
  • 1
    $\begingroup$ I would disagree with 1. Simply using a procedure because it allows you to make a decision isn't really a good reason, especially when you have no reason to expect that procedure to be valid. If you have little or no data then just make a judgement and don't pretend it's based on rigor. $\endgroup$ – dsaxton Jul 30 '15 at 13:13
  • 5
    $\begingroup$ I agree with you. It is clearly one reason why results are often not reproducible, even if published in highest ranked journals. But what options do you have as a researcher if budget allows only tiny sample sizes? $\endgroup$ – Michael M Jul 30 '15 at 13:28
  • 1
    $\begingroup$ Re 4 and the problems applying Wilcoxon-Mann-Whitney when there are unequal variances between groups, there are non-parametric methods that allow for heteroscedasticity: I seem to recall Cliff's test or the Brunner-Munzel test, for instance. (I don't think we have a lot of information about them on this site.) $\endgroup$ – Silverfish Jul 30 '15 at 22:55
  • $\begingroup$ @Silverfish: I am frequently using Brunner's methods and I think you are right. But I doubt that they are really comparing means exept when making strong distributional assumptions. $\endgroup$ – Michael M Jul 31 '15 at 7:36
  • 1
    $\begingroup$ @MichaelM Yes of course - that comes back to the question of the parametric and non-parametric methods having different hypotheses, I guess. $\endgroup$ – Silverfish Jul 31 '15 at 7:40
9
$\begingroup$

In hypothesis testing nonparametric tests are often testing different hypotheses, which is one reason why one can't always just substitute a nonparametric test for a parametric one.

More generally, parametric procedures provide a way of imposing structure on otherwise unstructured problems. This is very useful and can be viewed as a kind of simplifying heuristic rather than a belief that the model is literally true. Take for instance the problem of predicting a continuous response $y$ based on a vector of predictors $x$ using some regression function $f$ (even assuming that such a function exists is a kind of parametric restriction). If we assume absolutely nothing about $f$ then it's not at all clear how we might proceed in estimating this function. The set of possible answers that we need to search is just too large. But if we restrict the space of possible answers to (for instance) the set of linear functions $f(x) = \sum_{j=1}^{p} \beta_j x_j$, then we can actually start making progress. We don't need to believe that the model holds exactly, we are just making an approximation due to the need to arrive at some answer, however imperfect.

$\endgroup$
  • $\begingroup$ Yes, and that adds a model bias. What does that tell about the p-values researchers reporting? $\endgroup$ – Cowboy Trader Jul 30 '15 at 14:22
  • $\begingroup$ @dsaxton what you say is true about testing different hypotheses but people still interpret them the same way. Then there's also regression, where the insights you get between nonparametric and parametric analyses are pretty much the same. $\endgroup$ – en1 Jul 30 '15 at 14:41
  • $\begingroup$ @cagdas-ozgenc It tells you that p-values are conditional on the model. But it's not clear how things could be otherwise... $\endgroup$ – conjugateprior Jul 30 '15 at 14:46
  • 3
    $\begingroup$ +1 for noting that we have absolutely no chance of estimating anything of value in a regression without some assumptions about the regression function. $\endgroup$ – conjugateprior Jul 30 '15 at 14:47
8
$\begingroup$

Semiparametric models have many advantages. They offer tests such as the Wilcoxon test as a special case, but allow estimation of effect ratios, quantiles, means, and exceedance probabilities. They extend to longitudinal and censored data. They are robust in the Y-space and are transformation invariant except for estimating means. See http://biostat.mc.vanderbilt.edu/rms link to course handouts for a detailed example/case study.

In contrast to fully parametric methods ($t$-test, ordinary multiple regression, mixed effect models, parametric survival models, etc.), semiparametric methods for ordinal or continuous $Y$ assume nothing about the distribution of $Y$ for a given $X$, not even that the distribution is unimodal or smooth. The distribution may even have severe spikes inside it or at the boundaries. Semiparametric models assume only a connection (e.g., exponentiation in the case of a Cox model) between distributions for two different covariate settings $X_{1}$ and $X_{2}$. Examples include the proportional odds model (special case: Wilcoxon and Kruskal-Wallis) and proportional hazards model (special case: log-rank and stratified log-rank test).

In effect, semiparametric models have lots of intercepts. These intercepts encode the distribution of $Y$ nonparametrically. This doesn't, however, create any problem with overparameterization.

$\endgroup$
  • 1
    $\begingroup$ I've been struggling with this one a bit. Do you consider the t-test to be semi-parametric or non-parametric? On one hand, I've always assumed the 'jist' of semi-parametrics is the following: take a "working" probability model for the data, estimate parameters in that distribution regardless of whether the distribution is correct, and improve error estimation to account for uncertainty. (So sandwich based errors for solutions to Gaussian score equations would be a semi-parametric T-test). However, semi-parametrics almost always involve partialing/conditioning something out, as with Cox models. $\endgroup$ – AdamO Jul 30 '15 at 20:43
  • $\begingroup$ I'll add more description to my answer to deal with that. $\endgroup$ – Frank Harrell Jul 30 '15 at 21:05
6
$\begingroup$

The only reason I am answering despite all the fine answers above is that no one has called attention to the #1 reason we use parametric tests (at least in particle physics data analysis). Because we know the parametrization of the data. Duh! That's such a big advantage. You're boiling down your hundreds, thousands or millions of data points into the few parameters that you care about and describe your distribution. These tell you the underlying physics (or whatever science gives you your data).

Of course, if you don't have any idea of the underlying probability density then you have no choice: use non-parametric tests. Non-parametric tests do have the virtue of lacking any preconceived biases, but can be harder to implement - sometimes much harder.

$\endgroup$
5
$\begingroup$

Nonparametric statistics has its own problems! One of them is the emphasis on hypothesis testing, often we need estimation and confidence intervals, and getting them in complicated models with nonparametrics is --- complicated. There is a very good blog post about this, with discussion, at http://andrewgelman.com/2015/07/13/dont-do-the-wilcoxon/ The discussion leads to this other post, http://notstatschat.tumblr.com/post/63237480043/rock-paper-scissors-wilcoxon-test, which is recommended for a very different viewpoint on Wilcoxon. The short version is: the Wilcoxon (and other rank tests) can lead to nontransitivity.

$\endgroup$
  • 4
    $\begingroup$ I'm not sure that transitivity is the end-all and be-all. And you can invert the Wilcoxon test to get a very robust and useful location estimator confidence interval. $\endgroup$ – Frank Harrell Jul 30 '15 at 20:27
  • 2
    $\begingroup$ Nontransitivity has its points in causal modeling, but for simple two sample tests of association, I don't think it's really an issue. Also, I fail to see the differences in hypothesis testing/estimation/confidence intervals between nonparametric and parametric methods. Sometimes, with robust estimation, you use a working probability model so that the corresponding parametric estimation provides a meaningful summary of the data (even if it is not the right probability model per se). Perhaps you can expand on this answer? $\endgroup$ – AdamO Jul 30 '15 at 20:34
  • 2
    $\begingroup$ Usually some extra assumptions are made for the Wilcoxon test, e.g. stochastic dominance of one group over the other, that if true recover transitivity. $\endgroup$ – Scortchi Jul 31 '15 at 10:45
5
$\begingroup$

Among the host of answers supplied, I would also call attention to Bayesian statistics. Some problems cannot be answered by likelihoods alone. A Frequentist uses counterfactual reasoning where the "probability" refers to alternate universes and an alternate universe framework makes no sense as far as inferring the state of an individual, such as the guilt or innocence of a criminal, or whether bottlenecking of gene frequency in a species exposed to a massive environmental shift led to its extinction. In the Bayesian context, probability is "belief" not frequency, which can be applied to that which has already precipitated.

Now, the majority of Bayesian methods require fully specifying probability models for the prior and the outcome. And, most of these probability models are parametric. Consistent with what others are saying, these need not be exactly correct to produce meaningful summaries of the data. "All models are wrong, some models are useful."

There is, of course, nonparametric Bayesian methods. These have a lot of statistical wrinkles and, generally speaking, require nearly comprehensive population data to be used meaningfully.

$\endgroup$
3
$\begingroup$

I would say that non-parametric statistics are more generally applicable in the sense that they make less assumptions than parametric statistics.

Nevertheless, if one uses a parametric statistics and the underlying assumptions are fulfilled, then the paramatric statistics will be more powerfull than the non-parametric.

$\endgroup$
2
$\begingroup$

Parametric statistics are often ways to incorporate external [to data] knowledge. For instance, you know that the error distribution is normal, and this knowledge came from either prior experience or some other consideration and not from the data set. In this case, by assuming normal distribution you are incorporating this external knowledge into your parameter estimates, which must improve your estimates.

On your watch analogy. These days almost all watches are water resistant except for specialty pieces with jewelry or unusual materials like wood. The reason to wear them is precisely that: they're special. If you meant water proof then many dress watches are not water proof. The reason to wear them is again their function: you wouldn't wear a diver watch with a suite and tie. Also, these days many watches have open back so you can enjoy looking at the movement through the crystal. Naturally, these watches are usually not water proof.

$\endgroup$
  • 1
    $\begingroup$ I like this metaphor!. I remember one professor told us we should try different statistic methods on the same thing to see if we can get same results. $\endgroup$ – Deep North Jul 31 '15 at 0:13
2
$\begingroup$

This is not hypothesis testing scenario, but it may be a good example for answering your question: let's consider clustering analysis. There are many "non-parametric" clustering methods like hierarchical clustering, K-means etc., but the problem is always how to assess if your clustering solution is "better", than other possible solution (and often there are multiple possible solutions). Each algorithm gives you the best it can get, however how you know if there isn't anything better..? Now, there are also parametric approaches to clustering, so called model-based clustering, like Finite Mixture Models. With FMM you build a statistical model describing the distribution of your data and fit it into data. When you have your model, you can assess how likely is your data given this model, you can use likelihood ratio tests, compare AIC's, and use multiple other methods for checking model fit and model comparison. Non-parametric clustering algorithms just group data using some similarity criteria, while with using FMM enable you to describe and try to understand your data, check how good does it fit, make predictions... In practice non-parametric approaches are simple, work out-of-the box and are pretty good, while FMM can be problematic, but still, model-based approaches often provide you with richer output.

$\endgroup$
2
$\begingroup$

Predictions and forecasting to new data are often very difficult or impossible for non-parametric models. For example, I can forecast the number of warranty claims for the next 10 years using a Weibull or Lognormal survival model, however this is not possible using the Cox model or Kaplan-Meier.

Edit: Let me be a little more clear. If a company has a defective product then they are often interested in projecting the future warranty claim rate and CDF based on current warranty claims and sales data. This can help them decide whether or not a recall is needed. I don't know how you do this using a non-parametric model.

$\endgroup$
  • 6
    $\begingroup$ I beg to differ. You can use a Cox model to estimate quantiles, mean (if highest Y value is uncensored) and all sorts of probabilities. Not being able to forecast beyond the range of the data is a problem though (as you mentioned) but you could be dangerously extrapolating. $\endgroup$ – Frank Harrell Jul 30 '15 at 22:37
  • $\begingroup$ @FrankHarrell Yes good point, always must be careful when extrapolating. $\endgroup$ – Glen Jul 30 '15 at 22:40
  • $\begingroup$ What about Random Forest, Deep Learning, or SVM? They beat most if not all parametric methods for prediction. $\endgroup$ – en1 Jul 31 '15 at 14:31
  • 1
    $\begingroup$ Except try and make a decision tree learn a diagonal boundary $\endgroup$ – bill_e Jul 31 '15 at 16:07
1
$\begingroup$

I honestly believe that there is no right answer to this question. Judging from the given answers, the consensus is that parametric tests are more powerful than nonparametric equivalents. I won't contest this view but I see it more as a hypothetical rather than factual viewpoint since it is not something explicitly taught in schools and no peer reviewer will ever tell you "your paper was rejected because you used non-parametric tests". This question is about something that the world of statistics is unable to clearly answer but has taken for granted.

My personal opinion is that the preference of either parametric or nonparametric has more to do with tradition than anything else (for lack of a better term). Parametric techniques for testing and prediction were there first and have a long history, so it's not easy to completely ignore them. Prediction in particular, has some impressive nonparametric solutions which are widely in use as a first choice tool nowadays. I think this is one of the reasons that Machine Learning techniques such as neural networks and decision trees, which are nonparametric by nature, have gained widespread popularity over the recent years.

$\endgroup$
  • 2
    $\begingroup$ "the consensus is that parametric tests are more powerful than nonparametric": Non-parametric tests generally have lower power when the assumptions of the parametric test are correct, essentially since those assumptions mean parametric tests have a headstart (additional information about the true distribution). But especially in large samples the loss of power can be small: Wilcoxon is $3/\pi \approx 95\%$ as efficient as t-test in large samples if normality holds. If the assumptions are not met, non-parametric tests can be far more powerful! $\endgroup$ – Silverfish Aug 1 '15 at 9:12
  • $\begingroup$ 'Consensus' means 'general agreement', not my personal view. $\endgroup$ – Digio Aug 11 '15 at 9:39
  • 2
    $\begingroup$ I wasn't referring to whether the statement represented your own personal view or the collective wisdom of others, just pointing out that the statement is only correct if the necessary conditions for a parametric test hold true. When the conditions do not hold, it may no longer be the case that "parametric tests are more powerful than nonparametric", and in fact the reverse can be the case (sometimes by a very wide margin). $\endgroup$ – Silverfish Aug 11 '15 at 15:01
  • $\begingroup$ Touché! ..... +1 $\endgroup$ – Digio Aug 19 '15 at 12:19
0
$\begingroup$

It is a an issue of statistical power. Non-parametric tests generally have lower statistical power than their parametric counterparts.

$\endgroup$
  • 6
    $\begingroup$ Parametric tests will have more power when their assumptions are met. When their assumptions are not met, non-parametric tests can be more powerful. $\endgroup$ – gung Jul 30 '15 at 22:48
  • 3
    $\begingroup$ This answer's very short, & power has been discussed in previous answers. Would you consider expanding it a little? $\endgroup$ – Scortchi Jul 31 '15 at 8:45
  • 4
    $\begingroup$ The power gain from parametric tests is miniscule when compared to the power loss they suffer when their assumptions are not met. $\endgroup$ – Frank Harrell Jul 31 '15 at 14:17
  • $\begingroup$ Frank, it depends on the test, some tests are more robust to violations of their assumptions. $\endgroup$ – Hidden Markov Model Jul 31 '15 at 18:17

protected by John Dec 12 '15 at 3:25

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.