Suppose I have a generic model in state-space form described as

$$x_{t+1}=\phi_{t} x_{t}+w_{t+1}\epsilon_{t+1}$$ $$y_{t}=H_{t}x_{t}+v_{t}e_{t+1}$$

where both $e_{t+1}$ and $\epsilon_{t+1}$ are iid orthogonal white nose. Notice that all the parameters are time-varying. Suppose I know all the time-changing parameters (so I don't have to estimate them), except for $H_{t}$ and $v_{t}$, that are the time-changing parameters to be estimated.

Generally, in literature, I see that they use the Expectation Maximization algorithm to estimate the time-changing unknown parameters (a procedure like this pag. 17), which involves updating the estimates of the time-varying matrixes. However, why at least theoretically, not numerically, can't I use a simple MLE and define the estimated time-changing parameters as those set of matrixes $H_{t}$ and $v_{t}$ for t=1,...,T (where T is my sample size) that maximize the likelihood? Is there any theoretical countergument to do this? I am interested in a theoretical countergument, not a numerical one.


  • 1
    $\begingroup$ E-M is an algorithm for calculating the MLE. As a consequence, the only reason to choose a different algorithm for calculating the MLE is numeric, e.g., runtime. $\endgroup$ – jbowman Aug 13 at 19:14
  • $\begingroup$ So you are saying like "numerically it may be a better choice to use EM algorithm, but, analytically, MLE is fine (because in the end they are both maximizing a likelihood, although EM is a bayesian approach while MLE isn't)". Right? $\endgroup$ – Fr1 Aug 13 at 19:18
  • $\begingroup$ I don't think you've quite understood my point. EM is not a Bayesian approach. It's just an algorithm for efficiently finding the maximum of certain types of functions, including the likelihood function in missing-data problems. When you apply EM to such a problem, you get the MLE. If you apply some other maximization algorithm to such a problem, you will get the same MLE (numerical issues aside.) There is no "vs" between MLE and EM. One is the objective (maximize the likelihood), the other is a tool to get to the objective (calculate the parameters that maximize the likelihood.) $\endgroup$ – jbowman Aug 13 at 19:27
  • $\begingroup$ Yes, got it.. sorry for the Bayesian used improperly, it was just to denote a difference (anyway I quote this from Wikipedia "EM is a partially non-Bayesian, maximum likelihood method. Its final result gives a probability distribution over the latent variables (in the Bayesian style) together with a point estimate for θ (either a maximum likelihood estimate or a posterior mode)".. that is why I said so. $\endgroup$ – Fr1 Aug 13 at 19:36
  • $\begingroup$ Huh, "partially non-Bayesian"... I wonder what that means? Well I can see the source of your usage, thanks! $\endgroup$ – jbowman Aug 13 at 19:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.