3
$\begingroup$

Each iteration of the EM algorithm will lead to a (weak) increase in likelihood. I however get tiny decreases of the (estimated) likelihood just before the parameter estimates converge. This is the first time that I use the EM algorithm, so I would be interested if this phenomeon can appear with a correctly specified and coded method or not and, if yes, if it is a common phenomenon.

Could it just be that the algorithm keeps searching in a flat area of the likelihood around the maximum, which leads to numerical decreases in the likelihood approximation? What are further diagnostics that I could check?

To get a picture, a typical sequence of changes in expected likelihood per observation that I get in simulated data looks something like this:

3.7311e-06
1.706e-06
1.4912e-05
9.1953e-05
0.00047164
0.0019663
0.0074215
0.021332
0.035066
0.046388
0.057377
0.062306
0.057089
0.040042
0.019623
0.0064888
0.0018022
0.00049184
0.00013265
3.4079e-05
7.7135e-06
1.1234e-06
-2.5607e-07
-3.8119e-07
-2.7317e-07
-1.6555e-07
-9.3877e-08
-5.1608e-08
-2.7939e-08
-1.501e-08
-8.0357e-09
-4.2965e-09
-2.2974e-09
-1.2294e-09
-6.5877e-10
-3.5356e-10
-1.9011e-10
-1.0242e-10
-5.5285e-11
-2.9929e-11
-1.6223e-11
-8.828e-12
-4.7957e-12
-2.6277e-12
-1.4402e-12
theta converged -- exiting

This question is a follow-up to that question.

$\endgroup$
0