I don't think there can be one answer to all the deep learning models. WHich of the deep learning models are parametric and which are non-parametric and why?

  • 3
    $\begingroup$ Ask yourself a question: is the number of parameters increasing as you process new training examples during training? If so, the method is non-parametric. $\endgroup$ Jan 8, 2018 at 8:01
  • 3
    $\begingroup$ Given your first sentence, wouldn't your second sentence be a better title? $\endgroup$
    – Glen_b
    Jan 8, 2018 at 9:05
  • $\begingroup$ Could you define what you mean by parametric and non-parametric? Its not as clear cut as some might assume, see this wiki: en.wikipedia.org/wiki/Nonparametric_statistics for an example of the ambiguity $\endgroup$
    – Repmat
    Jan 8, 2018 at 13:08

4 Answers 4


Deep learning models are generally parametric - in fact they have a huge number of parameters, one for each weight that is tuned during training.

As the number of weights generally stays constant, they technically have fixed degrees of freedom. However, as there are generally so many parameters they may be seen to emulate non-parametric.

Gaussian processes (for example) use each observation as a new weight and as the number of points goes to infinity so too do the number of weights (not to be confused with hyper parameters).

I say generally because there are so many different flavours of each model. For example low rank GPs have a bounded number of parameters which are inferred by the data and I'm sure someone has been making some type of non-parametric dnn at some research group!

  • $\begingroup$ most people are confused that generally, NNs are Non-para models because they do some kind feature selection as the data points increase, hilarious!, then Linear regression with L1- regularization is also Non-parametric, what a disaster..!. Thank you, @j__ for answering it clearly. I didn't find a clear & concise answer anywhere so far. If you see any weblinks/research paper on this topic please post it here, would great to read more. $\endgroup$
    – Anu
    Nov 24, 2018 at 18:23
  • $\begingroup$ I think this answer is good, but it could benefit strongly by a definition of what the answer means by non-parametric. As the wiki-page on nonparametric statistics says, there is no general consensus on what non-parametric means. $\endgroup$
    – LudvigH
    Mar 5, 2021 at 10:00

A standard deep neural network (DNN) is, technically speaking, parametric since it has a fixed number of parameters. However, most DNNs have so many parameters that they could be interpreted as nonparametric; it has been proven that in the limit of infinite width, a deep neural network can be seen as a Gaussian process (GP), which is a nonparametric model [Lee et al., 2018].

Nevertheless, let's strictly interpret DNNs as parametric for the rest of this answer.

Some examples of parametric deep learning models are:

  • Deep autoregressive network (DARN)
  • Sigmoid belief network (SBN)
  • Recurrent neural network (RNN), Pixel CNN/RNN
  • Variational autoencoder (VAE), other deep latent Gaussian models e.g. DRAW

Some examples of nonparametric deep learning models are:

  • Deep Gaussian process (GPs)
  • Recurrent GP
  • State space GP
  • Hierarchical Dirichlet process
  • Cascaded Indian Buffet process

spectrum of latent variable models

Image from Shakir Mohamed's tutorial on deep generative models.



Deep learning models should not be considered parametric. Parametric models are defined as models based off an a priori assumption about the distributions that generate the data. Deep nets do not make assumptions about the data generating process, rather they use large amounts of data to learn a function that maps inputs to outputs. Deep learning is non-parametric by any reasonable definition.

  • $\begingroup$ Training deep learning models has weak a priori assumptions about the input data distribution: many hyper-parameters have defaults for - and are tuned for - standardised data. Does this count? $\endgroup$ May 11, 2023 at 3:39

Deutsch and Journel (1997, pp. 16-17) opined on the misleading nature of the term "non-parametric". They suggested that ≪...the terminology "parameter-rich" model should be retained for indicator based models instead of the traditional but misleading qualifier "non-parametric".≫

"Parameter rich" may be an accurate description, but "rich" has an emotional loading that lends a positive view which may not always be warranted (!).

Some professors may yet persist who refer collectively to neural nets, random forests, and the like as all being "non-parametric". The increased opacity and piecewise nature of neural nets (especially with the spread of ReLU activation functions) makes them non-parameteric-esque.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.