Applied probability is an important branch in probability, including computational probability. Since statistics is using probability theory to construct models to deal with data, as my understanding, I am wondering what's the essential difference between statistical model and probability model? Probability model does not need real data? Thanks.
$\begingroup$
$\endgroup$
0
asked Jun 23, 2012 at 18:40
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
11
A Probability Model consists of the triplet $(\Omega,{\mathcal F},{\mathbb P})$, where $\Omega$ is the sample space, ${\mathcal F}$ is a $\sigma$−algebra (events) and ${\mathbb P}$ is a probability measure on ${\mathcal F}$.
Intuitive explanation. A probability model can be interpreted as a known random variable $X$. For example, let $X$ be a Normally distributed random variable with mean $0$ and variance $1$. In this case the probability measure ${\mathbb P}$ is associated with the Cumulative Distribution Function (CDF) $F$ through
$$F(x)={\mathbb P}(X\leq x) = {\mathbb P}(\omega\in\Omega:X(\omega)\leq x) =\int_{-\infty}^x \dfrac{1}{\sqrt{2\pi}}\exp\left({-\dfrac{t^2}{2}}\right)dt.$$
Generalisations. The definition of Probability Model depends on the mathematical definition of probability, see for example Free probability and Quantum probability.
A Statistical Model is a set ${\mathcal S}$ of probability models, this is, a set of probability measures/distributions on the sample space $\Omega$.
This set of probability distributions is usually selected for modelling a certain phenomenon from which we have data.
Intuitive explanation. In a Statistical Model, the parameters and the distribution that describe a certain phenomenon are both unknown. An example of this is the familiy of Normal distributions with mean $\mu\in{\mathbb R}$ and variance $\sigma^2\in{\mathbb R_+}$, this is, both parameters are unknown and you typically want to use the data set for estimating the parameters (i.e. selecting an element of ${\mathcal S}$). This set of distributions can be chosen on any $\Omega$ and ${\mathcal F}$, but, if I am not mistaken, in a real example only those defined on the same pair $(\Omega,{\mathcal F})$ are reasonable to consider.
Generalisations. This paper provides a very formal definition of Statistical Model, but the author mentions that "Bayesian model requires an additional component in the form of a prior distribution ... Although Bayesian formulations are not the primary focus of this paper". Therefore the definition of Statistical Model depend on the kind of model we use: parametric or nonparametric. Also in the parametric setting, the definition depends on how parameters are treated (e.g. Classical vs. Bayesian).
The difference is: in a probability model you know exactly the probability measure, for example a $\mbox{Normal}(\mu_0,\sigma_0^2)$, where $\mu_0,\sigma_0^2$ are known parameters., while in a statistical model you consider sets of distributions, for example $\mbox{Normal}(\mu,\sigma^2)$, where $\mu,\sigma^2$ are unknown parameters.
None of them require a data set, but I would say that a Statistical model is usually selected for modelling one.
-
2$\begingroup$ @HonglangWang That is correct to some extent. The main difference is that a probability model is only one (known) distribution, while a statistical model is a set of probability models; the data is used to select a model from this set or a smaller subset of models that better (in a certain sense) describe the phenomenon (in the light of the data). $\endgroup$– user10525Commented Jun 23, 2012 at 20:52
-
3$\begingroup$ (+1) This is a nice answer, though I have a couple of comments. First, I think this may be selling the probabilist a little bit short. It is not at all uncommon to consider a set of probability spaces in a probabilistic model, and indeed, the possible measures can even be random (constructed on a suitably larger space). Second, a Bayesian (in particular) might find this answer slightly disconcerting in that a Bayesian statistical model can often be viewed as a single probability model on a suitable product space $\Omega \times \Theta$. $\endgroup$– cardinalCommented Jun 24, 2012 at 1:04
-
2$\begingroup$ @gung This a more measure-theory-related question. Regarding your first question, ${\mathbb P}$ is indeed defined through the CDF. Now, the interpretation of $\Omega$ is the difficult one because, formally, ${\mathbb P}(X\leq x)$ means ${\mathbb P}(\omega\in\Omega: X(\omega)\leq x)$, then $\Omega$ are not observable values. ${\mathcal F}$ is a $\sigma-$algebra which is the pre-image of the Borel $\sigma-$algebra under $X$, again this are not observable. I am not sure how to explain this in an intuitive level. $\endgroup$– user10525Commented Jun 24, 2012 at 20:04
-
3$\begingroup$ @gung $\Omega$ depends on the application; it is not determined by theory. For instance, $\Omega$ could be a set of Brownian motions describing the price of a financial derivative and $X$ could be the value attained at a fixed time $t$. In another application $\Omega$ could be a set of people and $X$ could be the lengths of their forearms. Generally, $\Omega$ is a mathematical model of the physical objects of study and $X$ is a numerical property of those objects. $\mathcal{F}$ is the set of possible events: those situations to which we want to ascribe probabilities. $\endgroup$– whuber ♦Commented Jul 18, 2012 at 18:51
-
3$\begingroup$ @gung $\mathcal{F}$ is a sigma algebra: it's a collection of subsets (the "events"). In the financial application, it's a set of price histories; in the forearm measurements application, the events would be sets of people. We can talk about this more if you want in a chat room. $\endgroup$– whuber ♦Commented Jul 18, 2012 at 20:01