I was reading these lecture notes and came accross the definition of GLM using exponential family of distribution. The latter seemed to have a bit of ambiguity, so I've checked that's it indeed the same in other soruces, so here is my question.

One says the the distribution on $\Bbb R$ is from exponential family if its density (PDF) or probability mass function (PMF) can be expressed as follows: $$ p(y|\theta)= b(y)\mathrm{exp}(\theta \cdot T(y) -a(\theta)). \tag{1} $$ Here comes my confusion: how can we treat PDFs and PMFs on par in such case. Of course, it is possible to do formally - PDFs and PMFs are just special cases of Radon-Nikodym derivatives of the probability distributions w.r.t. Lebesgue measure and counting measure respectively. However, it does not seem that the definition of exponential family really takes care of these things. For example, a given $p$ of the shape $(1)$ can be interpreted both as PDF and as PMF, leading to completely different probability distributions. This is not to say that $p$ can be as well interpreted as a RN derivative w.r.t. some other measure different from Lebesgue or counting, thus adding even more ambgiuity to this definition. Am I missing something?


1 Answer 1


I'm not sure what you're missing except maybe the support: An exponential family is defined with a density or mass function and a support $\Omega$.

You can find a measure-theoretic definition of exponential families in: Shao, J. (2003). Mathematical Statistics. Springer. http://books.google.ca/books?id=cyqTPotl7QcC

He writes:

A parametric family $\{P_\theta: \theta \in \Theta\}$ dominated by a $\sigma$-finite measure $\nu$ on $(\Omega, \mathcal F)$ is called an exponential family if and only if

$$\frac{dP_\theta}{d\nu} = \exp\left({[T(\omega)]}^\intercal \eta(\theta) - \xi(\theta)\right)h(\omega), \qquad \omega \in \Omega,$$

where $\exp(x) = e^x$ is the exponential function, $T$ is a random $p$-vector with a fixed positive integer $p$, $\eta$ is a function from $\Theta$ to $\mathcal R^p$, $h$ is a nonnegative Borel function on $(\Omega, \mathcal F)$, and $\xi(\theta) = \log\left(\int_\Omega > e^{{[T(\omega)]}^\intercal \eta(\theta)}h(\omega)d\nu(\omega)\right)$.

(I changed his notation to write dot products as $x^\intercal y$ rather than his $x y^\intercal$, which looks to me more like an outer product.)

I think that $\nu$ then could be your counting measure in the case of a p.m.f., or a Lebesgue measure for a p.d.f., etc.

  • $\begingroup$ Excellent, I'll read the whole chapter I guess - hopefully it does answer my question. $\endgroup$
    – Ulysses
    Dec 22, 2014 at 10:51
  • 1
    $\begingroup$ @Ulysses: Ok, please let us know if it answers your question. $\endgroup$
    – Neil G
    Dec 22, 2014 at 11:46
  • $\begingroup$ Ah, I thought that the acceptance of your answer implies that :) It answers my question, indeed. $\endgroup$
    – Ulysses
    Dec 23, 2014 at 16:25
  • $\begingroup$ It's not about having a "support", it's about having the reference measure $\nu$. The book excerpt doesn't even use the word "support". $\endgroup$
    – user357269
    Nov 16, 2021 at 9:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.