Suppose is is stated that $X_i \sim \mathcal{P}_X$ is a non-parametric model. What does this actually imply? I read in a book that sometimes one can specify a non-parametric model. It stated that:
$X_i \sim \mathcal{P}_X$ for $i = \{1, \ldots, n\}$ was a non-parametric model. 
My understanding of a non-parametric model is that it has an infinite-dimensional parameter space. I am wondering how to link it to the formula above, would it be possible?
 A: Without further context, I'm afraid that makes no sense.  When you are talking about the difference between "parametric" and "non-parametric" models, you are effectively referring to some underlying class of distributions.  In the parametric case the distribution sufficiently narrow in scope that it can be indexed by a finite number of real parameters (usually between one to four parameters in practice) and in the non-parametric case the class of distributions is so broad in scope that it can approximate any distribution up to arbitrary accuracy.  As you say, this might indeed involve an infinite dimensional real "parameter".  (That is usually what is meant by "non-parametric", although in some cases it might be a little narrower than this.)
If a book says $X_i \sim \mathcal{P}_X$, presumably the author intends for $\mathcal{P}_X$ the be the distribution of $X$.  This notation appears to refer to a single distribution, not a class of distributions, so as I said, without further context it makes no sense to say that it is "non-parametric".  (In a trivial sense, every individual distribution function is "non-parametric" insofar as it is not indexed by a parameter, but that is not what is usually meant when we talk about non-parametric models.)  I suspect that this is just bad wording on the part of the author --- i.e., he is saying it is non-parametric because he is referring to an individual distribution rather than a class of distributions indexed by a parameter.
A: I think $\mathcal{P}_X$ is generically referring to the probability distribution of the $X_i$, whatever it is. And I read the word "non-parametric" pretty literally. That is, we are not making the assumption that the distribution of $X$ is determined by finitely-many parameters. Whatever $\mathcal{P}_X$ is, it is not a distribution whose form can be summarized by a finite set of numbers.
Say that $X$ is a univariate random variable and you want to estimate its probability density function $f_X(x)$. The parametric approach is to assume that the pdf belongs to a family of distributions with finitely-many parameters (the gaussian family, the gamma family, the poisson family, etc). Given that assumption, your task is to estimate the parameters to determine the entire density function. If you do not make that assumption, you are working non-parametrically. You are still trying to estimate $f_X(x)$, but without making a parametric assumption about its functional form. In that case, you are trying to estimate the function value $f_X(x)$ for any value of $x$. For a continuous random variable, there are (uncountably) infinitely-many $x$ values, so you have infinitely-many function values $f_X(x)$ to estimate. So when you say "[m]y understanding of a non-parametric model is that it has an infinite-dimensional parameter space," in this case it means that the "parameter" you are estimating is the entire function, which is an infinite-dimensional object. But the word "parameter" is a bit of a misnomer at that point.
