Density estimation vs. (proper) density estimation or parameter estimation

As far as I can tell, given observations coming from an unknown distribution $p$ (purported to belong to some class $\mathcal{C}$ of distributions, say e.g. mixtures of Gaussians), there are 3 possible "goals":

• if $\mathcal{C}$ is parametric: estimate the parameters of $p$ ("parameter estimation")

• estimate $p$ (say, in $L_2$, KL or $L_1$ distance) by a density $\hat{p}\in\mathcal{C}$ ("proper learning")

• estimate $p$ (say, in $L_2$, KL or $L_1$ distance) by a density $\hat{p}$ not necessarily in $\mathcal{C}$, e.g. from a superclass $\mathcal{H}\supseteq \mathcal{C}$ ("improper learning")

Intuitively, these tasks range from the hardest to the simplest, and the last two are equivalent as long as computational efficiency is not required.

My question is: what is the motivation for the third one? I.e., in what practical or semi-applied setting would it be beneficial to have in one's hands a good approximation of the distribution with say a kernel estimate, while the original is assumed to be a mixture of 3 Gaussians and an exponential?

To be clear: i understand the theoretical challenges to achieve even the third task, and that in some cases the first two are either too hard or seem hopeless, so tackling (3) is at least a good first step. My question is: in what situations would a practitioner be satisfied with the improper density learning, and what would that practitioner do with the hypothesis $\hat{p}\notin\mathcal{C}$?

Edit:

Motivations I can think of are:

• being able to approximately predict the probability of any possible event (and given the type of hypothesis, compute them easily) -- useful e.g. for risk analysis (?)

• being able to generate more samples roughly distributed as they "should." This could be very useful in experimental sciences: while having few "real" samples from sensors, one can then still be able to run models and simulations requiring many more -- by generating them from the hypothesis (this, of course, requires the said models to be themselves robust).

PS: references or pointers to back up possible applications would be great.