A strictly $k$-parameter exponential family $$f=\exp\left(\sum \eta_i(\theta)T_i(x)-B(\theta)\right)h(x)$$ is regular if the natural parameter space $\eta(\theta)$ contains a $k$-dimensional open set.

I was given this counterexample but couldn't figure out why it is not regular: $$\exp\left[\sum_{j=1}^{k-1} \left(\log \theta_j−\log\left(1−\sum_{l=1}^{k-1}\theta_l\right)\right)N_j+n\log\left(1−\sum_{l=1}^{k-1}\theta_l\right)\right].$$ For reference this is the likelihood of $n$ i.i.d. random variables $x_i$, taking value $j\in \{1,...,k\}$ with probability $\theta_j$, and $N_j=\sum_i1_{x_i=j}$.

And in general, how do I check if an exponential family is regular?

  • $\begingroup$ Should that last $=j$ be part of the subscript? $\endgroup$ Jan 5 at 23:47
  • $\begingroup$ yes I've fixed that, thanks $\endgroup$
    – orangecat
    Jan 6 at 0:16

1 Answer 1


The easy way to see this one isn't regular is that the $\theta_j$ need to add up to 1, so (given sufficient smoothness) they can only fill up at most a $k-1$-dimensional set.

Somewhat more generally, you could work out the information matrix and see that it is of rank less than $k$ everywhere in the interior of the parameter space. I don't know if that's a necessary condition to be non-regular, though.

(To make the calculations easier, you might start with $j=2$)

  • $\begingroup$ Thanks for the response, but I'm still a bit confused - I thought k-1 dimensional set wasn't problematic because it's a strictly k-1 parameter family? $\endgroup$
    – orangecat
    Jan 6 at 0:24
  • 1
    $\begingroup$ The $\theta_j$ in the OP's counterexample actually don't need to add to one because there are $k-1$ of them instead of $k$. The trouble is that the representation in the counterexample is not the natural parameter space, for which your statement is correct. $\endgroup$
    – jbowman
    Jan 6 at 2:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.