Skip to main content
added 1412 characters in body
Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k

A p-value is not the value of a pdf, but an actual probability.

Wikipedia has it right:

In statistical significance testing the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.

That is, a p-value might be found from an appropriately defined cdf, rather than a pdf.

Let's now consider - assuming we have a sample from the right cdf - whether a kernel density estimate computed on that sample is appropriate for computing p-values.

There are a number of relevant points I can make:

i) the estimate of the CDF obtained from a kernel density estimate is smoother than the original ECDF; indeed smoothness (of the PDF estimate) is the point of kernel density estimates

ii) the kernel density estimate has larger variance than the thing it estimates; it's not an unbiased estimate of the PDF. The CDF estimate will therefore also have bias.

iii) kernel density estimates are "optimized" for estimating PDFs, not CDFs. If you want an good smooth estimate of a CDF, you may be better directing effort to optimizing desirable properties of that.

iv) in the univariate case, often the preference seems to be to use the sampled values themselves to estimate empirical quantiles (though it is sometimes done); I don't know that this appearance of a preference carries over to multivariate cases.

In aummary: Yes, it's legitimate in the sense that - done correctly - it might provide reasonable estimates of p-values. It will be biased (tend to give too high a p-value in the tails and correspondingly too low in the middle). You should take care to try to optimize your smoothing to the purpose you are putting it to.

A p-value is not the value of a pdf, but an actual probability.

Wikipedia has it right:

In statistical significance testing the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.

That is, a p-value might be found from an appropriately defined cdf, rather than a pdf.

A p-value is not the value of a pdf, but an actual probability.

Wikipedia has it right:

In statistical significance testing the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.

That is, a p-value might be found from an appropriately defined cdf, rather than a pdf.

Let's now consider - assuming we have a sample from the right cdf - whether a kernel density estimate computed on that sample is appropriate for computing p-values.

There are a number of relevant points I can make:

i) the estimate of the CDF obtained from a kernel density estimate is smoother than the original ECDF; indeed smoothness (of the PDF estimate) is the point of kernel density estimates

ii) the kernel density estimate has larger variance than the thing it estimates; it's not an unbiased estimate of the PDF. The CDF estimate will therefore also have bias.

iii) kernel density estimates are "optimized" for estimating PDFs, not CDFs. If you want an good smooth estimate of a CDF, you may be better directing effort to optimizing desirable properties of that.

iv) in the univariate case, often the preference seems to be to use the sampled values themselves to estimate empirical quantiles (though it is sometimes done); I don't know that this appearance of a preference carries over to multivariate cases.

In aummary: Yes, it's legitimate in the sense that - done correctly - it might provide reasonable estimates of p-values. It will be biased (tend to give too high a p-value in the tails and correspondingly too low in the middle). You should take care to try to optimize your smoothing to the purpose you are putting it to.

Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k

A p-value is not the value of a pdf, but an actual probability.

Wikipedia has it right:

In statistical significance testing the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true.

That is, a p-value might be found from an appropriately defined cdf, rather than a pdf.