# Parameters in a non-parametric model

I have not understood this Wikipedia statement:

The difference between parametric model and non-parametric model is that the former has a fixed number of parameters, while the latter grows the number of parameters with the amount of training data.

How does the number of parameters in a non-parametric model increase with the amount of training data?

• Hi. Did you get to see a practical implementation of this, where number of parameters are growing with new incoming data? It would be a huge help if you could point me to such a source. I have just started nonparametrics(HDP) and am struggling with relating the theory to practical. – rj dj Mar 25 '19 at 16:22

Nonparametric methods don't specify something (maybe the distribution, maybe a relationship between two variables) with a fixed finite number of parameters; they're (potentially) infinite parametric.

Consider a loess curve (a form of nonparametric regression), for example; the parameters not only aren't explicit, if you attempt to count them, the number isn't even an integer.

On the other hand, you never need more than $n$ parameters to define $n$ observations; presumably in at least that sense* the number of parameters grows as you increase $n$.

*(though in other senses as well)

Consider, for example, a kernel density estimate using the usual AIMSE-optimal bandwidth selection; that has an effective number of parameters (e.g. as measured by Ye's generalized degrees of freedom) that grows as $n$ increases (but not in proportion to $n$).

However, since the statement is referenced (Murphy, Kevin (2012). Machine Learning: A Probabilistic Perspective. MIT), you should probably consult that work for the complete context.

• I have been just introduced with non-parametric. In non-parametric method , is there any way to understand when only $1$ parameter is enough and when i need more than one parameter(2,3,etc) ? – time Feb 14 '15 at 6:42
• I wouldn't read too much into the statement, nor worry over much about precisely how many parameters are required in any particular situation. The model is (at least potentially) infinite-parametric -- but these parameters - outside of the ones you're immediately testing, if any - need not be (and usually aren't) explicit. – Glen_b -Reinstate Monica Feb 14 '15 at 6:47
• Consider, for example, a Kolmogorov-Smirnov two sample test. No explicit parameters are involved. It doesn't matter whether the sample sizes are (10,15) or (85,120) -- the test works the same both times; the test statistic is the largest difference in ECDFs and while the model could be regarded as having $n_1+n_2$ parameters in that particular instance, that's of no particular interest. – Glen_b -Reinstate Monica Feb 14 '15 at 6:54
• It is written that in a parametric model all the parameters are in finite-dimensional parameter spaces. What do they mean by finite-dimensional parameter spaces? In normal distribution, $-\infty<\mu<\infty$. It doesn't range finitely. – time Feb 20 '15 at 13:11
• It's not the domains of the parameters, but the dimension of the parameter vector itself. In the case of a univariate normal, the parameter vector is $(\mu,\sigma)^\top$, which is a vector of dimension 2. The boundedness or unboundedness of the values $\mu$ or $\sigma$ can take aren't under consideration. – Glen_b -Reinstate Monica Feb 20 '15 at 14:15