2
$\begingroup$

I am working on some applications of time series, and I wanted to find a book that has the numerical algorithms or pseudocode for computing things like AR models, and ARIMA models, using nonlinear least squares, method of moments, the differencing operations, as well as State-space model like Kalman filters, etc. I can use all of the existing packages in python or R, etc., but I would rather write the code myself so that I understand it.

I have copies of Shumway and Stoffer and all of the other books, but they just show the mathematics without showing estimation algorithms. Even the videos I have found on line don't really give much in the way of estimation.

Ideally the book or reference would show how to generate a fake time-series of data, then show how to difference the data and estimate the coefficients. I can figure out most of this myself, but just seems like there should be a better reference for stuff like this. Thanks.

Improve this question
$\endgroup$
11
  • $\begingroup$ Hi" if you're really interested in that sort of thing, it's a huge undertaking. Suppose you're interested in using R to do it. Then you will need to go into the source of one of the arima functions ( say the one in base ) and go line by line through the code to understand it. There are many pitfall-subtleties when dealing with arima estimation ( KF estimation is slightly easier but no joke either ) so it will be a difficult journey. The reason it's not easy to find in a text is because it's quite difficult numerically and to do it correctly takes serious programming expertise . $\endgroup$
    – mlofton
    May 26, 2020 at 6:23
  • $\begingroup$ @mlofton thanks for the note. Your point is quite good. I was able to find a pretty good treatment in the book by Kitagawa INTRODUCTION TO TIME SERIES MODELLING, which actually provides some of the numerical aspects as well as the model. I might just put this as the answer. That book does not provide the code, but it does lay out the algorithms in a more understandable way. The KF chapter is not that great, but there are many online KF tutorials. I will try and code up the KF for an ARMA likelihood myself and get it to work :). Wish me luck. $\endgroup$
    – krishnab
    May 28, 2020 at 19:03
  • $\begingroup$ @mlofton of course none of these books really discuss the practical numerical issues with computing for statistics or ML. And most numerical methods courses focus on root finding, interpolation, linear systems, and then ODE/PDEs. In stats we always see simple likelihood based methods for proofs, but for any slight complexity or larger dataset the likelihood methods become intractable or take too long to compute. Haha, if engineers can program KFs, then stats folks should be able too as well :). $\endgroup$
    – krishnab
    May 28, 2020 at 19:08
  • $\begingroup$ I think you should actually go into the code of an existing working function that is open source and see how it works. This would be true whether you want to program the ARIMA function or the KF function. For example, in the case of the ARIMA function, you need to check whether your estimates result in a polynomial that has zeroes inside the unit circle. If not, then you have to chuck them and keep searching. Things like that are probably not going to be found in most textbooks. I don't think you need pdes-odes or linear systems. Good luck. $\endgroup$
    – mlofton
    May 28, 2020 at 22:20
  • $\begingroup$ Another way of looking at the above is that there are constraints on the parameters that are implicit. For example, in the case of the ARIMA(0,0,2), instead of working with the zeros of the polynomial, you can convert the polynomial constraint into an equivalent constraint on the parameters. Box-Jenkins has an example of this for the ARIMA(0,0,2). $\endgroup$
    – mlofton
    May 28, 2020 at 22:24

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.