2
$\begingroup$

I have a little perplexity trying to distinguish parametric vs non-parametric generative model.

In my understanding, a parametric generative model would try to learn the probability density function by estimating the parameters of an underlying distribution we are assuming. So just doing for example,

$$\theta^* = arg\max_\theta \,\prod_{i=1}^N p_\theta(\textbf{x}_i)$$

I realize that in practice, we need to figure out what is the basic distribution that we are going to modify by adjusting the parameters $\theta$. So in the case of VAEs we use latent variables assumption to make training feasible, we jointly train $q_\phi(\textbf{z}|\textbf{x})$ and $p_\theta(\textbf{x}|\textbf{z})$ prametrizing both distributions with neural networks (i.e. encoder & decoder). In such case, we end up with the situation that all our distributions are Gaussians (assuming that prior and conditional are gaussians). So, having said that, can we conclude that VAEs are parametric? Also, what could be an example of non-parametric generative model?

I would say that, for example, GANs maybe an example of non-parametric model, as we start with a latent normal distribution but then applying a stack of non-linear transformations ending up with something potentially very complicated.

$\endgroup$
1
  • $\begingroup$ Non-linear transformations are so often parametric. $\endgroup$
    – Alexis
    Jul 12, 2022 at 16:07

1 Answer 1

1
$\begingroup$

Parametric: no matter what training set you are using, your model occupies the same amount of memory in your pc (ie has the same amount of parameters)... Neural Net are parametric (otherwise good luck optimizing them)

Non parametric: the size of your models may vary, usually depends on the size of the training set

In other words: the "parametric" property of a model does not depend on the structure of the model (ie the NN architecture) but on how the model is defined on the very basis"

SVM and Decision trees are usually used as examples for non parametric models, however they are not usually considered generative

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.