I am working with some two-regime autoregressive models first introduced by Hamilton in 1989. The specific models is of no great concern to my question, but some variables within my autoregressive models are latent binary variables - non-observable binary variables, that is. I have a dataset and use MCMC to find posterior densities of all the parameters of my models, including all the latent binary variables.

I have a dataset of about 1000 observations, and for every observation I have a latent variable in my models, which can wither be 1 or 0. If my data suddendly changes, I can assume that a switch has been turned on and the latent variable has changed. So if the data, at a specific time, is (most likely) best described by the latent variable being 1, my sampler will produce a posterior distribution (for this specific latent variable) with high probability of 1 and low probability of 0.

So this is all good, I have managed to do this. The question is, how do I validate my model? AIC/BIC is just for comparing models. What I am thinking is that I have to be able to plot the residuals somehow, like in normal regression where you just say that the residual is the difference of observed and predicted result. I understand that for all other parameters than the latent parameters/variables, i can just use the mean of each posterior distribution and treat it like a Maximum Likelihood Estimator. But I cant take the mean of my posterior distributions for all the latent variables, because they are either 0 or 1. It does not make any sense using, for example, 0.8 for one of my latent variables. This is no option.

So how should I go on to validate my model? I am really stuck here. If I have done a bad job explaining my problem, I apologize and will try to explain better.

So I have been thinking. Can I use a predictive posterior distribution to validate my model? And if too many observations, lies above, say, the 95 percentile of the predictive posterior distribution, I throw away my model?

Generally, for any model whose parameters are called theta, observations called x, and new observation called x_new, one calculate the predictive posterior distribution as follows: For i = 1, . . . , n (samples), we sample: 1. theta[i] from p(theta|x) and 2. x_new[i] from p(x_new|theta[i]) Then x_new[1], . . . ,x_new[n] are a sample from the predictive posterior distribution. But this method must surely only work when we have a iid sample. I am working with an autoregressive model. So I am stuck...

  • $\begingroup$ For example, a model could look like this: Y_t=alpha + beta_1*X_tY_(t-1) + beta_2*X_(t-1)*Y_(t-2) +sigma_tepsilon_t , where the X_t and X_(t-1) are two latent variables either 1 or 0, alpha,beta's,sigma are parameters, and epsilon_t is standard normally distributed. So this is an autoregressive model, with lag=2. $\endgroup$ – Erosennin Oct 15 '13 at 19:04
  • $\begingroup$ If I wanted to get the residuals, I could choose the mean of the posterior distributions of the alpha,beta_1,beta_2 and sigma_t and get something like: r_t= (Y_observed - Y_predicted)/sigma_t = standard normally distributed, but I would have to choose some value for the latent variables. And this makes no sense. $\endgroup$ – Erosennin Oct 15 '13 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.