2
$\begingroup$

In Bishop's Pattern Recognition and Machine Learning, he states that exponential family distributions over $\mathbf{x}$ given parameters $\boldsymbol{\eta}$ can be written as $p(\mathbf{x} | \boldsymbol{\eta}) = h(\mathbf{x})g(\boldsymbol{\eta})\exp\{\boldsymbol{\eta}^\top\mathbf{u}(\mathbf{x})\}$.

He gives the natural parameters for the Bernoulli distribution with probability of success $\mu$ as: $$\eta = \ln(\frac{\mu}{1 - \mu})$$ $$u(x) = x$$ $$h(x) = 1$$ $$g(\eta) = \frac{1}{1 + \exp{(\eta)}}$$

I was working this out myself and found $$\boldsymbol{\eta} = [\ln\mu,\: \ln(1 - \mu)]^\top$$ $$\mathbf{u}(x) = [x, \: 1-x]^\top$$ $$h(\mathbf{x}) = 1$$ $$g(\boldsymbol{\eta}) = 1$$ could also work.

How do you know when you've found the natural parameters and is there a rule preventing you from adding arbitrary scaling constants into the expressions for $\boldsymbol{\eta}$, $\mathbf{u}$, $h$, and $g$?

$\endgroup$
  • 1
    $\begingroup$ there's a typo, it must be $u(x)=x$ $\endgroup$ – Christoph Hanck Jan 14 '16 at 16:38
  • 1
    $\begingroup$ ...and unless I am mistaken, also $g(\eta) = \frac{1}{1 + \exp{(\eta)}}$ $\endgroup$ – Christoph Hanck Jan 14 '16 at 16:44
3
$\begingroup$

A natural exponential family is one in which the natural parameter $\eta$ and the statistic $u(x)$ are both the identity (see here), which is the case for Bishop's formulation, but not yours.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.