If aren't they synonymous, what distinguishes the one from the other? Is probability density estimation a certain kind of generative model? Can any generative model be regarded as density estimation?


Density estimation is generally seen as a nonparametric goal: rather than using specific families such as the Gamma family or the Normal family, density estimation procedures can arrive at any shape. I haven't seen anyone use the name "density estimation" for a parametric procedure, presumably because then it's just parameter estimation.

By contrast, generative models often have parametric components; for example, the Latent Dirichlet Allocation model uses the Dirichlet distribution.

  • $\begingroup$ Thanks. Here Prof. Ng introduces EM as a way of doing density estimation, wouldn't that be a parametric goal? youtu.be/ZZGTuAkF-Hw?list=PLA89DCFA6ADACE599&t=1279 $\endgroup$ – Lenar Hoyt Jan 12 '16 at 22:28
  • $\begingroup$ I guess it's a fuzzy boundary. Nonparametric procedures sometime use mixtures of Gaussians -- just with one Gaussian per data point. They might even be used generatively; that's sort of the idea behind the bootstrap. I would never call LDA "density estimation", even though it can yield a density estimate. $\endgroup$ – eric_kernfeld Jan 13 '16 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.