Are the terms Hypothesis Space and Parameter Space Interchangeable? In the Machine Learning literature there seems to be a term called the Hypothesis Space. As a statistician, I find this to be a bit of a awkward term, but in use it seems to imply that it is equivalent to a non-parametric parameter space (infinite dimensional). Are these two terms equivalent?
 A: No, they're not quite the same. In Machine Learning, the Hypothesis Space is the set of all models of reality that a learning algorithm is able to adapt to. Since the models are often not explicitly formulated, the Hypothesis Space might also not be rigorously defined. For a classifier algorithm, it would be the set of all distinct classifiers that the algorithm is able to generate. 
The closest equivalent in statistics would be the set of all possible statistical models that you willing to entertain as possibly true. In classical textbook parametric statistics, there might well be a one-to-one correspondence between elements of the parameter space and possible models so, in that case, you can essentially line up the two concepts.
In general though, "Hypothesis Space" is a looser and more inclusive concept. Suppose you are doing an applied statistical analysis using normal-theory linear regression but also using regression diagnostics to check for outliers and so on. Then the Hypothesis Space implied by your analysis procedure includes a wide variety of outlier models in addition to the classical normal models implied by the linear regression. The outlier models will not usually be explicitly defined but are merely implied by the methods you use to check for outliers and by the remedial action you take when you detect one.
So the Machine Learning people have given a name to something that exists in statistics but which we don't always make explicit. In simple cases it might be equivalent to the parameter space mapped onto the mathematical model that the parameters are part of.
