# Predicting a semi-deterministic process

Say I have a process that gives me 3 outputs: $O^1$, $O^2$ and $O^3$.

The outputs are generated from a semi-deterministic process, i.e. there is a deterministic component in the outputs, along with a random component.

In particular, having $n$ measurements over time, the outputs $O_j \quad j=1,2,...,n$ are -at least in part- dependent on the previous outputs. So $O_j = f(O_{j-1}, O_{j-2}, O_{j-3}) + \epsilon$ (I'm not interested in going farther away than 2 or 3 measurements, $\epsilon$ is the random component).

So now I have a set of ~150 consecutive measurements, how can I predict what are the likely outputs in the future?

I can easily calculate the distribution of values following a certain output, for instance I could say that if $O^1_j$ is between 50 and 60 I have a certain probability of $O^1_{j+1}$ of being between 30 and 40, by looking at the measurements that I took in the past. I did construct some pdf for the distribution of these probabilities, but now I'm a bit stuck, especially because there is probably an interaction between the three outputs (so for instance, updating my previous statement $O^1_j = f(O^1_{j-l}, O^2_{j-l}, O^3_{j-l}) + \epsilon \quad\quad l=1,2,3$)

I've been reading about Bayesian predictors and I tought they could be applied here, but I don't know enough about the topic to determine if this is a good choice or if there is something easier/more appropriate. I will appreciate any suggestion!

-
Have you considered vector-valued autoregressive processes as a model? en.wikipedia.org/wiki/Autoregressive_model . – whuber Jan 9 '11 at 17:51
Your second equation, $O^1_j = f(O^1_{j-l}, O^2_{j-l}, O^3_{j-l}) + \epsilon$, is confusing. Is this the same $f$ as previously? (I hope not, because that wouldn't make much sense.) What exactly do you mean by setting $l$ to 1, 2, 3? That seems to give three separate equations for $O_j^1$. – whuber Jan 9 '11 at 17:53
@whuber: No, the f is different (but note that it is unknown). You're right, probably the equation is confusing... I meant that essentially you have 9 parameters, $O^1$, $O^2$ and $O^3$ for the past 3 observations. I will have a look to the autoregressive models, I am not completely familiar with them but I guess it's a good pointer as a start. – nico Jan 9 '11 at 18:01
You may find it helpful, both conceptually and notationally, to think of $O$ as a sequence of vectors $(O_j^1, O_j^2, O_j^3)$ indexed by "time" $j$. You can then write $O_j = f(O_{j-1}, O_{j-2}, O_{j-3}) + \epsilon$ (and $\epsilon$ is a random vector with zero expectation). It would help immensely if you can assume $f$ is linear, at least to a good approximation: that's the AR model. (If this is a correct representation of the situation, then your last formula needs additional fixing...) – whuber Jan 10 '11 at 2:24

If you use R, then first step can be performed by function stl, it is function from standard R. Autoregressive models can be fit automatically by auto.arima in package forecast. This function can either fit your desired model, or find the best specification for certain definition of best. You might look into that package more, since it is specially designed for forecasting time series. For VAR model use VAR function from vars package. This package has a nice vignette describing its capabilities.