I am trying to write down the exact definition of a bayesian statistical model in a similar way as the definition for a statistical model.

So far I have the definition:

A statistical model is a pair $(X,(\mathbb{P}_{\theta})_{\theta\in\Theta})$, where $(\mathbb{P}_{\theta})_{\theta\in\Theta}$ is a family of probability distributions on a measurable space $(S,\mathcal{A})$. $X$ is a random variable mapping to $S$, whose distribution has a free parameter $\theta \in \Theta$. This means, we know a function $\theta \mapsto \rho_{\theta}$ with $$ P_{\theta}(X \in da) = \rho_{\theta}(a)da $$ \begin{align*} \text{with}&&P_{\theta}(X \in da) & := P(X=a) \text{, for }S\text{ discrete and} \\ &&P_{\theta}(X \in da) &:= f(a)da \text{, for density }f(a). \end{align*}

Now I wanted to add that in bayesian models the parameter $\theta$ is considered a random variable on its own:

Let $(S\times\Theta,\mathcal{A}\times\mathcal{B},P)$, with an appropriate $\mathcal{B}$, be a probability space and let $(X,T)$ be a random variable mapping to $(S\times\Theta)$. For the family $(\mathbb{P}_\theta)_{\theta\in\Theta}$ of conditional distributions $$\mathbb{P}_{\theta}(\cdot):=P(\cdot|T=\theta)$$ and the marginal distribution $$ \Pi:\mathcal{B}\rightarrow[0,1],\quad \Pi(B) := P(T\in B)= P((X,T)\in S\times B),$$ the statistical model $(X,\mathbb{P}_{\theta})$ is called a bayesian statistical model with A-Priori-distribution $\Pi$.

My problem/question is, that I know $\mathbb{P}_{\theta}(\cdot):=P(\cdot|T=\theta)$ is wrong, because it must be $$P(\cdot|T\in B), \quad B\subset \Theta$$ for it to be a conditional distribution because $\{T=\theta\}$ would be of measure zero if $\Theta$ is not discrete.

However, I am not sure whether the definition of $\mathbb{P}_{\theta}(\cdot)$ then still makes sense as in the end, the 'true parameter' which we want to find is a single value $\theta_0$ which produces the observations $X_1,...,X_n\sim\mathbb{P}_{\theta_0}$ ? I.o.w. how can I reconcile $\mathbb{P}_{\theta}\text{ and }P(\cdot|T\in B)\text{ ?}$

Thank you very much in advance!

  • 3
    $\begingroup$ I don't this is a question about Bayesian statistics as much as it is a question about conditional distributions. To make the question more succinct, you could ask, why, in a bivariate normal distribution $p(x,y)$ of $(X,Y)$, can we speak of the conditional distribution $p(y|X=x)$, considering that $X=x$ is an event with probability zero? $\endgroup$ Commented Jul 13, 2021 at 13:11
  • 1
    $\begingroup$ Perhaps stats.stackexchange.com/questions/230545/… addresses the issue? For more like this, use this site search. $\endgroup$
    – whuber
    Commented Jul 13, 2021 at 13:48
  • 1
    $\begingroup$ @whuber Thank you very much! I didn't know yet that conditioning on an event implicitly means conditioning on the sigma algebra generated by it, nor did I even know about conditioning on sigma algebras, so I will definitely look into it! $\endgroup$
    – klm
    Commented Jul 13, 2021 at 14:10


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.