Skip to main content
2 of 2
Commonmark migration

The question claims "when a method routinely produces high p-values it is called conservative." As pointed out by @Acccumulation in the comments, a p-value has a precise definition. One does not have more or less conservative p-values. In practice, sometimes one has to estimate a p-value (e.g. by using the bootstrap), and I suppose one could describe such an estimator as "conservative". But I haven't seen this in practice, and I don't think that's what the question is getting at.

Although I don't have a reference handy, it certainly seems natural to refer to one hypothesis test as being more conservative than another if it has a smaller type 1 error. Using liberal in the opposite sense seems possible, though I can't remembering seeing that anywhere.

The term "conservative" is often used for confidence intervals. A 95% confidence interval procedure will have different coverage probabilities depending on what the true value of the parameter is. For example, in Brown et al.'s Interval Estimation for a Binomial Proportion, speaking about two different confidence intervals for a Bernoulli probability p, they say "the coverage probability of the [Agresti–Coull] interval is quite conservative for p very close to 0 or 1. In comparison to the Wilson interval it is more conservative, especially for small n." Saying it's conservative for p very close to 0 or 1 means that for p close to 0 or 1, the probability of the interval containing the true value of p will be very high -- higher than the nominal coverage of the interval (say 95%).

DavidR
  • 1.8k
  • 12
  • 15