# Practical method to do MLE for natural parameters in exponential family

I encountered the following question in my research and I hope this is the correct place to post it. I'm following the notation in this lecture note by Michael I. Jordan.

Assume random vector $$X$$ follows some exponential family distribution:

$$p(X=x|\eta)=\exp(\eta^T T(x)-A(\eta))f(x),$$

where $$\eta$$ is the vector of natural parameters and $$T(x)$$ is the vector of sufficient statistics. As an exponential family, it can also be parametrized by the mean parameter vector:

$$\mu = \mathbb{E}_{p(x|\eta)}[T(X)] = \frac{\partial A}{\partial \eta^T}(\eta).$$

Let $$\mathbf{x}_n = (x_1,...,x_n)$$ be a data vector composed of i.i.d. samples from this distribution. I would like to do maximum likelihood estimation for the parameter $$\eta$$. It can be shown that the MLE of the mean parameter is:

$$\hat{\mu}_{ML} = \frac{1}{n}\sum_{i=1}^n T(x_i),$$

so mathematically we have:

$$\hat{\eta}_{ML} = \left(\frac{\partial A}{\partial \eta^T}\right)^{-1}(\hat{\mu}_{ML}).$$

However, in practice, usually $$A(\eta)$$ is only given by the normalization condition $$\int p(x|\eta) dx = 1$$ and it is not explicitly known. $$A(\eta)$$ itself is also hard to evaluate numerically because $$X$$ may be of high dimension. In view of this, I'm wondering in this case how should we get the value of $$\hat{\eta}_{ML}$$?

In most (standard) exponential families, the moment generating function $$A)\eta)$$ is available in closed form. Here is a screen-copy from Wikipedia that shows the beginning of a list of examples, each of which can be easily inverted:
• Thank you for your answer. The distribution I encountered doesn't fall into those standard exponential family distributions. Actually here the $X$ is of very high dimension, and I'm pretty confident that the closed-form expression for $A(\eta)$ is not known. – user166061 Feb 28 at 20:02
• It's a statistical mechanics problem and its details are not very related to this question. Also I think this question itself is also valid in a broader context. The only background information I want to borrow from the original problem are 1. $X$ is of high dimension and 2. the closed-form of $A(\eta)$ is not available. – user166061 Feb 28 at 21:02