6
$\begingroup$

The probability density function (pdf) is the first derivative of the cumulative distribution (cdf) for a continuous random variable. I take it that this only applies to well-defined distributions like the Gaussian, t-distribution, Johnson SU, etc, though.

If given real data that we know does not conform to some prior distribution (perfectly), does that mean that (it would be safe to assume that) the real data's cdf cannot be differentiated, and therefore has no pdf, making us resort to histogram, or kernel density, or log-spline approximations, of the continuous data's pdf?

just trying to rationalize the whole model-fitting craze (Gaussian, t-, Cauchy) that is always encountered in statistics, and why it always overrides approximation approaches (histogram, kernel density).

In other words, rather than use an estimator on the empirical data (histogram, kernel density), we are trained to look for a best match model (Gaussian, t-, Cauchy) instead, even though we know the real data's pdf diverges from that model.

What makes the "modeling" approach better than "approximation"? Is it, and how is it, more right?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Is your question: “why do we use parametric distributions instead of empirical distributions“? $\endgroup$
    – Tim
    Commented Sep 6, 2020 at 14:47
  • $\begingroup$ yes i should say that $\endgroup$
    – develarist
    Commented Sep 6, 2020 at 14:48
  • $\begingroup$ Convergence is faster for parametric distribution estimators than for non-parametric estimators, the higher the dimension the larger the advantage. $\endgroup$
    – Xi'an
    Commented Sep 6, 2020 at 15:25
  • $\begingroup$ @Xi'an i would like to see this advantage, and by how much the advantage is. is there a source that actually demonstrates how increasing sample size will show parametric distributions to be more accurate for real data than its empirical distribution as the empirical distribution also increases with sample size? $\endgroup$
    – develarist
    Commented Sep 6, 2020 at 15:56
  • $\begingroup$ We do use the ecdf -- e.g. in bootstrapping. $\endgroup$
    – Glen_b
    Commented Sep 6, 2020 at 16:32

1 Answer 1

Reset to default
14
$\begingroup$

An enormous amount of data is needed to accurately estimate a distribution nonparametrically, especially a continuous one. Even then, some assumptions about the smoothness of the distribution are needed for filling the gaps (interpolating) between the observed values and other assumptions are needed for extrapolating outside the observed data range. With a small or moderate sample, you would usually expect poor accuracy from a nonparametric estimation. It would take a large discrepancy between the true distribution and a modelled parametric one used to approximate it to make the nonparametric approach more accurate. This is especially true in higher dimensions, as data become sparser when the dimension grows.

Improve this answer
$\endgroup$
7
  • $\begingroup$ is there a source that actually demonstrates how increasing sample size will show parametric distributions to be more accurate for real data than its empirical distribution as the empirical distribution also increases with sample size? $\endgroup$
    – develarist
    Commented Sep 6, 2020 at 15:55
  • 4
    $\begingroup$ @develarist, I did not say increasing the sample size will make parametric distributions more accurate; it is rather the opposite. However, the sample sizes needed for accurate nonparametric estimation are very seldom. Also, the problem as you have formulated it is simply too broad to analyze; such general results may not be obtained. But you could show it for many separate, concrete cases. $\endgroup$
    – Richard Hardy
    Commented Sep 6, 2020 at 15:57
  • $\begingroup$ then there's no evidence to support. i'd also be interested by how much more accurate the parametric distribution is than the empirical distribution for low samples, and then for high number of samples. single out the Gaussian vs empirical case if we have to de-broaden $\endgroup$
    – develarist
    Commented Sep 6, 2020 at 16:00
  • 2
    $\begingroup$ @develarist, quite naturally there is no concrete evidence for many general claims, but each concrete case provides evidence that can be generalized. $\endgroup$
    – Richard Hardy
    Commented Sep 6, 2020 at 16:01
  • 1
    $\begingroup$ "An enormous amount of data is needed to accurately estimate a distribution nonparametrically, especially a continuous one" And more especially a continuous and multivariate one! $\endgroup$
    – leonbloy
    Commented Sep 7, 2020 at 2:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.