I encountered the following question in my research and I hope this is the correct place to post it. I'm following the notation in this lecture note by Michael I. Jordan.
Assume random vector $X$ follows some exponential family distribution:
$$p(X=x|\eta)=\exp(\eta^T T(x)-A(\eta))f(x),$$
where $\eta$ is the vector of natural parameters and $T(x)$ is the vector of sufficient statistics. As an exponential family, it can also be parametrized by the mean parameter vector:
$$\mu = \mathbb{E}_{p(x|\eta)}[T(X)] = \frac{\partial A}{\partial \eta^T}(\eta).$$
Let $\mathbf{x}_n = (x_1,...,x_n)$ be a data vector composed of i.i.d. samples from this distribution. I would like to do maximum likelihood estimation for the parameter $\eta$. It can be shown that the MLE of the mean parameter is:
$$\hat{\mu}_{ML} = \frac{1}{n}\sum_{i=1}^n T(x_i),$$
so mathematically we have:
$$\hat{\eta}_{ML} = \left(\frac{\partial A}{\partial \eta^T}\right)^{-1}(\hat{\mu}_{ML}).$$
However, in practice, usually $A(\eta)$ is only given by the normalization condition $\int p(x|\eta) dx = 1$ and it is not explicitly known. $A(\eta)$ itself is also hard to evaluate numerically because $X$ may be of high dimension. In view of this, I'm wondering in this case how should we get the value of $\hat{\eta}_{ML}$?
