3
$\begingroup$

The sample space of binomial distribution is the set $\{0,1\}$ and its $\sigma$-algebra is the power set of $\{0,1\}$ while the sample space of normal distribution is $\mathbb R$ and its $\sigma$-algebra is the Lebesgue measurable set. In these cases, the probability measure of binomial distribution is of the form of exponential family while the radon-nikodym derivative of the probability measure (i.e. pdf) of normal distribution is of the form of exponential family. Their natures are so different and why do people even call it a family?

My questions are:

  1. How do we determine if exponential family gives us the probability measure or the probability density function?
  2. How can we determine the sample space, $\sigma$-algebra and its probability measure from an exponential family?

Any help are welcome.

$\endgroup$
1
  • 2
    $\begingroup$ The sample space of a Binomial $\mathcal B(n,p)$ is $\{0,1,\ldots,n\}$. $\endgroup$
    – Xi'an
    Jul 29, 2021 at 3:14

1 Answer 1

5
$\begingroup$

Some elements of answer:

  • The pmf for a distribution with a countable sample space is its density wrt to the counting measure.
  • Exponential families are made of families of parametric distributions whose density wrt to a given dominating measure $\text d\lambda(x)$ write as $f_\theta(x)=\exp\{R(\theta)^\text{T}S(x)-\psi(\theta)\}$, where $\psi(\theta)$ is the log-normalising constant.$^1$ The Binomial and the Normal distributions belong to different exponential families.$^2$
  • the sample space, $\mathfrak X$, its σ-algebra $\mathfrak B(\mathfrak X)$ and the dominating measure $\text d\lambda(x)$ are components of the definition of a given exponential family, just as much as the functions $R(\cdot)$ and $S(\cdot)$ above. If they are not provided, the associated exponential family is not defined.$^3$

$^1$I purposedly did not include the traditional $h(x)$ in the representation of $f_\theta(x)$, i.e., $f_\theta(x)=h(x)\exp\{R(\theta)^\text{T}S(x)-\psi(\theta)\}$ to stress the fact that $h(\cdot)$ can as well be part of the dominating measure. Changing $h(\cdot)$ thus amounts to changing the dominating measure.

$^2$In particular, the Binomial distribution is absolutely continuous wrt the counting measure on $\mathfrak X=\mathbb N$ and the Normal distribution is absolutely continuous wrt the Lebesgue measure on $\mathbb R$

$^3$In other words, a probability distribution does not exist independently of its dominating measure, which itself does not exist independently of its σ-algebra $\mathfrak B(\mathfrak X)$, which itself requires a sample space $\mathfrak X$

$\endgroup$
6

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.