According to Wikipedia,

Given an observable variable X and a target variable Y, a generative model is a statistical model of the joint probability distribution on X × Y, P(X,Y)

However, as far as I know, the term "generative model" is used exclusively for machine learning models, e.g. generative neural networks. But there are also some non-machine learning models that can produce synthetic data, for example this model for synthetic ECG signal generation.

Can such non-machine-learning models be called "generative" as well?

If not, what is the correct terminology for them?

  • 1
    $\begingroup$ What would a "non-machine-learning model" be? $\endgroup$ Aug 2 '18 at 8:24
  • $\begingroup$ @Scortchi basically any mathematical model that is able to produce the "artificial" data. There is, for example, a model for synthetic ECG signal generation, which, in my opinion, could be called "generative". $\endgroup$
    – Kao
    Aug 2 '18 at 9:43
  • $\begingroup$ So any model that's not "learnt" from data, but that's come up with some other way? (Rather than an ML vs a Trad Stats model.) $\endgroup$ Aug 2 '18 at 10:39
  • $\begingroup$ @Scortchi Yes, exactly. $\endgroup$
    – Kao
    Aug 2 '18 at 11:05
  • $\begingroup$ Could you please edit your question to include that clarification? $\endgroup$ Aug 2 '18 at 12:31

Generative models are so called because they suffice to generate synthetic data, but the raison d'être of the term is to distinguish them from discriminative models (which don't, having discarded information on the marginal distribution of $X$ by conditioning). So there's no point calling a model "generative" unless you have that distinction in mind—unless the data comprise features/predictors/inputs/... ($X$) that you use to predict labels/responses/outputs/... ($Y$). Nevertheless, insofar as "machine-learning based" doesn't describe every stochastic model that you might use for prediction/classification, the answer to the question in your title is "no"—the provenance of the model isn't germane.


As far as I am aware the term generative model is used only for machine learning models that match the above definition. Mainly this is to differentiate them from discriminative models, which is also only applicable to machine learning models.

Can non-machine-learning models that produce data be called "generative" as well?

I don't recall seeing this expression before and would refer to it differently, although I suppose it would be understood anyway.

If we think of a statistical model as an idealized/simplified version of a data generating process, I'd assume it's ability to generate data is inherent and doesn't need to be explicitly expressed.

Edit: I admit that my gut got the better of me on that last paragraph, which I redact after some thought on the matter.

  • $\begingroup$ Please check this paper out. It lists 'A probability model must describe the generation of the data' as a frequent misconception and uses 'generative model' to describe a type of statistical model, not necessarily machine learning one. $\endgroup$
    – Kao
    Jul 24 '18 at 11:45
  • $\begingroup$ Thanks for the reference, I didn't have time yet to read it in detail. Coming back to my answer after a few days, I removed the last part, which I lack proof for. $\endgroup$
    – deemel
    Aug 2 '18 at 8:08
  • $\begingroup$ @kao: It's clear in context that that's a quite different use of "generative model", making another distinction: "We need a descriptive model, not necessarily a generative model." I suspect it's rather idiosyncratic: "mechanistic" or "theoretical" (distinguishing from "empirical" or "phenomenological") would be more common. $\endgroup$ Aug 3 '18 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.