I am interested in estimation of ARMA models. I understand that a popular approach is to write the model down in the state space form and then maximize the likelihood of the model using some optimization routine.

Question: Why rewrite the model into its state space representation and maximize the corresponding likelihood -- instead of maximizing the "naive" or "direct" likelihood?

(I could imagine that a different parameterization can make the optimization easier -- is that the case here?)

Related questions:

I am also aware of some general advantages and disadvantages of the state space representation as mentioned in "What are disadvantages of state-space models and Kalman Filter for time-series modelling?".

  • $\begingroup$ Simple reparametrizations are used in state space models in order to simplify estimation procedures as described in Durbin Koopman's and A. Harvey's books. Also in comparison to ARIMA, state space parametrization is richer as ARIMA parametrization is the reduced form of state-space and doesn't take into account the convergence to steady state. $\endgroup$ – Cowboy Trader Nov 1 '16 at 14:58
  • $\begingroup$ @CagdasOzgenc, oh, so then the change (improvement?) in estimation due to the state space (SS) representation is actually coming from reparameterization? That would explain some of my confusion. Could you give a very simple example of how the parameterization changes from direct likelihood to SS? E.g. in a simple AR(1) model I thought the model parameters are directly put into the SS without any changes (e.g. the slope $\varphi_1$ remains as is, it does not become some, say, $2\theta+1$ or whatever). But maybe I overlooked something. $\endgroup$ – Richard Hardy Nov 1 '16 at 15:29
  • $\begingroup$ Have you looked at these lecture notes: kris-nimark.net/pdf/Handout_S4.pdf ? Even if you write the model in state space form you will need to filter through the data using (e.g.) the Kalman filter. The prediction error can then be used to construct a likelihood function which needs to be maximized. So you will need to maximize a likelihood function one way or the other. $\endgroup$ – Plissken Nov 1 '16 at 16:01
  • 1
    $\begingroup$ @Plissken ...which is what I am trying to say here write the model down in the state space form and then maximize the likelihood of the model [due to that form] using some optimization routine. So I understand that likelihood maximization is inevitable. The question is, is the direct likelihood and the likelihood of the state space representation the same or different? And if they are different, is that due to reparameterization which makes one of the likelihood functions easier to maximize? $\endgroup$ – Richard Hardy Nov 1 '16 at 16:17
  • $\begingroup$ Not a full answer but: 1) alternatives don't always optimize the same likelihood, it's usually a conditional likelihood on the first few observations, whereas in state-space exact diffuse initialization is available, and 2) Kalman filter provides recursion for the likelihood naturally but also a recursion for the gradient as a by-product, which is useful for optimisation. $\endgroup$ – Chris Haug Nov 1 '16 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.