MLE of an exponential distribution in discrete case

Question

Consider the one parameter exponential family on a finite sample space $S$: $$p(x;\theta)=\frac{e^{\theta x}}{\sum_{x\in S} e^{\theta x}}, \theta\in \mathbb{R}.$$

My objective is to find an MLE for $\theta$ having observed a sample $x_1,\dots,x_n$ from $p(\cdot,\theta)$. In the continuous case (when $\theta$ is restricted to strictly negative and $x>0$) one can easily solve the likelihood equation and see that the MLE is negative of reciprocal of the sample mean.

When I do the same here, I obtain that the $\theta$ for which $p(x_1,\dots,x_n;\theta)$ is maximum satisfies $$\frac{\sum_x xe^{\theta x}}{\sum_x e^{\theta x}}=\frac{1}{n}\sum_i x_i.$$ Is it possible to find MLE from here in a closed form expression? LHS is mean with respect to $p(\cdot,\theta)$. Is the mean known in terms of $\theta$? Any help is appreciated.

Answer 1

In general there is no closed form solution. This can be seen by substituting $$u=e^\theta \hspace{2em}\text{and}\hspace{2em} z=\frac{1}{n}\sum_i x_i$$ and rearranging the equation which leads to $$\sum_x (x-z)\,u^x = 0$$ Under the assumption that the elements $x$ of the sample space are positive integers, this is a power series. A special case would be $S:=\{0,1,2\}$ where the solution can be found by solving the quadratic equation. Since it is known that equations with integral powers of order greater than four in general don't have a closed form solution also the equation in your question does not have one.