3
$\begingroup$

In Bishop's Pattern Recognition and Machine Learning, he states that exponential family distributions over $\mathbf{x}$ given parameters $\boldsymbol{\eta}$ can be written as $p(\mathbf{x} | \boldsymbol{\eta}) = h(\mathbf{x})g(\boldsymbol{\eta})\exp\{\boldsymbol{\eta}^\top\mathbf{u}(\mathbf{x})\}$.

He gives the natural parameters for the Bernoulli distribution with probability of success $\mu$ as: $$\eta = \ln(\frac{\mu}{1 - \mu})$$ $$u(x) = x$$ $$h(x) = 1$$ $$g(\eta) = \frac{1}{1 + \exp{(\eta)}}$$

I was working this out myself and found $$\boldsymbol{\eta} = [\ln\mu,\: \ln(1 - \mu)]^\top$$ $$\mathbf{u}(x) = [x, \: 1-x]^\top$$ $$h(\mathbf{x}) = 1$$ $$g(\boldsymbol{\eta}) = 1$$ could also work.

How do you know when you've found the natural parameters and is there a rule preventing you from adding arbitrary scaling constants into the expressions for $\boldsymbol{\eta}$, $\mathbf{u}$, $h$, and $g$?

$\endgroup$
2
  • 1
    $\begingroup$ there's a typo, it must be $u(x)=x$ $\endgroup$ Jan 14, 2016 at 16:38
  • 1
    $\begingroup$ ...and unless I am mistaken, also $g(\eta) = \frac{1}{1 + \exp{(\eta)}}$ $\endgroup$ Jan 14, 2016 at 16:44

1 Answer 1

3
$\begingroup$

A natural exponential family is one in which the natural parameter $\eta$ and the statistic $u(x)$ are both the identity (see here), which is the case for Bishop's formulation, but not yours.

$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.