Problem in discrete valued time series forecasting

Question

I have a temporally ordered discrete valued data. The only possible states for the data are: {1,2,3,4,5,6}. So the series is something like {1,2,3,5,6,4,3,5,2,......} I want to forecast the next value of the series based on past data.What are the methods appropriate for this?

Now please note that my question is not repetitive. I am aware of this

How to model categorical (discrete-valued) time series?

Wherein people suggested that a HMM or a Markov model would be appropriate. I searched the site for that, downloaded R packages like HiddenMarkov, hmm.discnp, etc. Esp in the later, I used the Viterbi function given and got the most likely sequence of hidden states that led to the generation of observed states, i.e. data given to it.

But that is not what I want! what I want is FUTURE value of the OBSERVED sequence from the PRESENT and PAST values and the Viterbi function does not give that. What is the solution for this?

Also, what other methods are useful for which some packages are available? I read about discrete ARMA methods, but not for multi-class data and then also without any implementation. Is this problem really that hard? Please suggest preferably a software implementation or a method.

Answer 1

Returning to the discarded HMM approach. An HMM is just a state space model that assumes discrete hidden state. I think you have dismissed the state space approach too soon.

The forecasting question you bring up is quite general for such models which I'll discuss in discrete time formulation, since that's what you've got. (If you happen to know about how Kalman filters work, use that intuition; they're used for very similar state space models when gaussian assumptions apply.)

State and the role of past observations

The basic assumption of any state space model is that the future is independent of the past given the current (hidden) state, and that the observations at a point depend only on the hidden state. Consequently when you have the state you don't need the preceding observed values or states. (And if that isn't enough, you need a formulation with more state...) The state is intended to represent all the relevant information.

Forecasting

Conceptually speaking, to forecast, you start with a distribution over the current hidden state, and you use the estimated transition function -- a transition matrix in the HMM case -- to propagate this distribution forward in time. Once you've moved forward to the point in time where you want to forecast you have a (wider) distribution over the future state. (Alternatively in the HMM case you can find the most likely state to be in then, but you will have neglected quite a bit of uncertainty.) You then generate a predictive distribution over observables by taking a weighted average of the conditional distribution of the observations where the weights are from the distribution over the hidden state.

Basically it's just like filtering, only without tightening up the state estimate on the basis of observations. On cursory glance, the packages you mention don't seem to have a function for more than one step ahead forecasting (because that's needed to compute the likelihood) but now you know what you want to do, you may be able to persuade them to do so, or write the relevant function yourself.

This is described in mathematical detail in a bunch of places, e.g ch. 5 of Zucchini and MacDonald (2008) for the HMM case. They also offer an R implementation. Petris and Patrone (2011) is a more general review of state space model packages in R, including those that work with non-discrete state models. These would be good places to start.

Multivariate state space modelling is not particularly straightforward, but most of the difficulties are numerical and good implementations for the classic models are increasingly available.

Answer 2

The airline series is a count series i.e the number of people flying per month. You could consider your series to be also a count series ( in reality it is an attribute/class series) but at a much lower level (1,2,3,4,5,6). Now if you were to identify and estimate an ARIMA model the problem arises insofar as the fitted values ( 1 period out predictions) have not been constrained to be integers which represents a slight problem. I once wrote software to constrain the fitted values to be integers but parameter convergence disappeared. I have studied rare disease data on a monthly basis that was similar to your "count series" and have been able to identify a useful seasonal patterns yielding a prediction which will be a non-integer. Rounding the forecast to an integer serves the purpose of providing predictions that will be integers. One can identify level shifts and/or local time trends in this kind of data and also to some extent anomalies. What I am suggesting is imperfect but until someone tells me a better way , here I stand. I would be more than happy to provide you with an example using existing software if you posted an actual historical series.

REVISED With Data analysis:

The plot of the 34 data points and the ACF of the data suggest that the data might be daily as the ACF of lag 7 suggests structure. A model was automatically developed which developed an auto-regressive memory of order 2 and an indicator variable for day 1 of each week with two Pulses ( anomalies) being identified at periods 14 and period 11 . The Actual and Cleansed plot illuminate the two unusual data points . . The residuals from the model appear visually to be "random" and the ACF of the Residuals appears to support that conclusion . The actual/fit/forecast plot is shown here . This model can be expressed as Pure Right-Hand Side ala a regression equation Note that when predicting the Pulse Indicators play no role as future values of these indicators are expected to be 0 while the future value of the first day of the week will be zero except for the first day of the week. For example if we are predicting out 14 days from period 34 , this would mean 0,1,0,0,0,0,0,1,0,0,0,0,0,0 .Note that all models are wrong but some models are useful ( attributed to G.E.P.Box ) .

THE RIGHT-HAND SIDE CONSTANT IS: 3.5646 Y 1 .649306 Y( 34 )= 6.000000 3.895837 Y 2 -.539719 Y( 33 )= 5.000000 -2.698596 NET PREDICTION FOR Y( 35 )= 4.761866

The above is a prediction for period 35. Note that you would have to round off the forecasts in order to meet the integer requirements. Thus the prediction would be "5". I hope this helps. In closing here is a summary of the model .