# Is EM feasible when there is no closed form maximization of the expectation of log likelihood?

In every example I've seen of expectation maximization, the E step concludes with an expression of the expectation of log likelihood ( $Q(\theta | \theta^{(t)})$ ) for which a maximum w.r.t. $\theta$ can be determined easily in closed form using

$\frac{\partial Q(\theta | \theta^{(t)})}{\partial \theta} = 0$.

However, I am working on a model where the expectation of the log likelihood is quite complex, and cannot be maximized in closed form expressions. I am wondering whether it's feasible to estimate the maximum numerically as part of the algorithm.

Questions:

1. When there is no closed form maximization of $Q(\theta | \theta^{(t)})$, can one be determined numerically in code?
2. Are there any good examples of work where this is done that I can study?

The maximization step can be conducted by numerical optimization. In fact, it is possible to incorporate constraints (if any) on the original problem into the numerical optimization by use of constrained numerical optimization. One possible use of such constraints is to steer the whole EM optimization away from "bad" regions; but they can also be used to incorporate hard constraints on parameter values.

In fact, the maximum need not even be found during the maximum step. In a variant called Generalized Expectation Maximization, the M step may consist of only one or some number of optimization algorithm iterations, with the M step being terminated prior to a maximum being found. The key is to get an improvement in the objective function in each M step.

If numerical optimization of the M step isn't easy or effective enough, you may be better off using a traditional numerical optimizer on the full problem. All sorts of hybrid approaches are possible.