One-Step ahead predictive likelihood for time series forecasting

I am still new to Bayesian forecasting, so I am hoping to get some clarification on a simple concept (by the sounds of it).

Suppose that we are interested in forecasting some time series one-step ahead. For simplicity, we can assume the time series itself came from a normal distribution with known parameters. The one-step-ahead predictive likelihood (as I understand) is:

$PL = p(y^{o}_t|y^{o}_{t-1}) = \int_\theta p(y_{t}^o|y^o_{t-1},\theta)\times p(\theta|y^o_{t-1})d\theta$

Which can be approximated by $M^{-1}p(y^{o}_{t}|y^{o}_{t-1},\theta^{(m)}_{t-1})$, for $m$ MCMC loops.

How is it possible to calculate $p(y^{o}_t|y^{o}_{t-1})$?! It's the probability from a normal distribution (i.e. 0). Unless this has some relationship to the likelihood being evaluated at some point, I have no idea what to do here or evaluate this with a given posterior.

Most Bayesian time series models are based on dynamic models. For simplicity, let's consider the random walk with noise model.

$$y_t = x_{t\phantom{-1}} + v_t \qquad v_t \sim N(0,V) \\ x_t = x_{t-1} + w_t \qquad w_t \sim N(0,W)$$

and $x_0\sim N(m_0,C_0)$. Since we have "known parameters", we know the values for $V$ and $W$. The one-step ahead predictive distribution is

$$p(y_t|y_{t-1}) = \int \int p(y_t|x_t)p(x_t|x_{t-1})p(x_{t-1}|y_{1:t-1}) dx_t dx_{t-1}$$

Let $x_{t-1}|y_{1:t-1} \sim N(m_{t-1}, C_{t-1})$ where $y_{1:t-1}$ is all the data up to time $t-1$. Then the one-step ahead predictive distribution is $y_t|y_{t-1} \sim N(m_{t-1}, C_{t-1}+W+V)$.

Other dynamic linear models (with known parameters) will also have a normal one-step ahead predictive distribution. Non-linear models or linear models with non-normal errors will generally not have a closed form predictive distribution. If parameters are unknown, you will also have to integrate over your current uncertainty in those parameters, i.e. $p(\theta|y_{1:t-1})$ and this will also typically not have a closed form predictive distribution.

• This is not the predictive likelihood however? This is a pdf. Or are you suggesting to determine the likelihood of $y_{t}|y_{t-1}$
– akkp
Commented Feb 22, 2015 at 21:56
• I'm not sure what you mean by "predictive likelihood" since you state "known parameters" and you write $p(y_t|y_{t-1})$, so I assumed you were wondering about a one-step ahead predictive distribution for your data. Commented Feb 23, 2015 at 16:13