Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Applied probability is an important branch in probability, including computational probability. Since statistics is using probability theory to construct models to deal with data, as my understanding, I am wondering what's the essential difference between statistical model and probability model? Probability model does not need real data? Thanks.

share|improve this question
I think you should reword the first sentence, as such, it is rather abusive... –  Xi'an Jun 25 '12 at 0:45
I modified a little bit. Thanks very much. –  Honglang Wang Jun 25 '12 at 1:08
add comment

1 Answer

up vote 20 down vote accepted

A Probability Model consists of the triplet $(\Omega,{\mathcal F},{\mathbb P})$, where $\Omega$ is the sample space, ${\mathcal F}$ is a $\sigma$−algebra (events) and ${\mathbb P}$ is a probability measure on ${\mathcal F}$.

Intuitive explanation. A probability model can be interpreted as a known random variable $X$. For example, let $X$ be a Normally distributed random variable with mean $0$ and variance $1$. In this case the probability measure ${\mathbb P}$ is associated with the Cumulative Distribution Function (CDF) $F$ through

$$F(x)={\mathbb P}(X\leq x) = {\mathbb P}(\omega\in\Omega:X(\omega)\leq x) =\int_{-\infty}^x \dfrac{1}{\sqrt{2\pi}}\exp\left({-\dfrac{t^2}{2}}\right)dt.$$

Generalisations. The definition of Probability Model depends on the mathematical definition of probability, see for example Free probability and Quantum probability.

A Statistical Model is a set ${\mathcal S}$ of probability models, this is, a set of probability measures/distributions on the sample space $\Omega$.

This set of probability distributions is usually selected for modelling a certain phenomenon from which we have data.

Intuitive explanation. In a Statistical Model, the parameters and the distribution that describe a certain phenomenon are both unknown. An example of this is the familiy of Normal distributions with mean $\mu\in{\mathbb R}$ and variance $\sigma^2\in{\mathbb R_+}$, this is, both parameters are unknown and you typically want to use the data set for estimating the parameters (i.e. selecting an element of ${\mathcal S}$). This set of distributions can be chosen on any $\Omega$ and ${\mathcal F}$, but, if I am not mistaken, in a real example only those defined on the same pair $(\Omega,{\mathcal F})$ are reasonable to consider.

Generalisations. This paper provides a very formal definition of Statistical Model, but the author mentions that "Bayesian model requires an additional component in the form of a prior distribution ... Although Bayesian formulations are not the primary focus of this paper". Therefore the definition of Statistical Model depend on the kind of model we use: parametric or nonparametric. Also in the parametric setting, the definition depends on how parameters are treated (e.g. Classical vs. Bayesian).

The difference is: in a probability model you know exactly the probability measure, for example a $\mbox{Normal}(\mu_0,\sigma_0^2)$, where $\mu_0,\sigma_0^2$ are known parameters., while in a statistical model you consider sets of distributions, for example $\mbox{Normal}(\mu,\sigma^2)$, where $\mu,\sigma^2$ are unknown parameters.

None of them require a data set, but I would say that a Statistical model is usually selected for modelling one.

share|improve this answer
@HonglangWang That is correct to some extent. The main difference is that a probability model is only one (known) distribution, while a statistical model is a set of probability models; the data is used to select a model from this set or a smaller subset of models that better (in a certain sense) describe the phenomenon (in the light of the data). –  user10525 Jun 23 '12 at 20:52
(+1) This is a nice answer, though I have a couple of comments. First, I think this may be selling the probabilist a little bit short. It is not at all uncommon to consider a set of probability spaces in a probabilistic model, and indeed, the possible measures can even be random (constructed on a suitably larger space). Second, a Bayesian (in particular) might find this answer slightly disconcerting in that a Bayesian statistical model can often be viewed as a single probability model on a suitable product space $\Omega \times \Theta$. –  cardinal Jun 24 '12 at 1:04
@gung This a more measure-theory-related question. Regarding your first question, ${\mathbb P}$ is indeed defined through the CDF. Now, the interpretation of $\Omega$ is the difficult one because, formally, ${\mathbb P}(X\leq x)$ means ${\mathbb P}(\omega\in\Omega: X(\omega)\leq x)$, then $\Omega$ are not observable values. ${\mathcal F}$ is a $\sigma-$algebra which is the pre-image of the Borel $\sigma-$algebra under $X$, again this are not observable. I am not sure how to explain this in an intuitive level. –  user10525 Jun 24 '12 at 20:04
@gung $\Omega$ depends on the application; it is not determined by theory. For instance, $\Omega$ could be a set of Brownian motions describing the price of a financial derivative and $X$ could be the value attained at a fixed time $t$. In another application $\Omega$ could be a set of people and $X$ could be the lengths of their forearms. Generally, $\Omega$ is a mathematical model of the physical objects of study and $X$ is a numerical property of those objects. $\mathcal{F}$ is the set of possible events: those situations to which we want to ascribe probabilities. –  whuber Jul 18 '12 at 18:51
@gung $\mathcal{F}$ is a sigma algebra: it's a collection of subsets (the "events"). In the financial application, it's a set of price histories; in the forearm measurements application, the events would be sets of people. We can talk about this more if you want in a chat room. –  whuber Jul 18 '12 at 20:01
show 6 more comments

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.