I have some data I need to analyze, and some prior knowledge I'd like to apply. I have discrete time points, and noisy, continuous outputs $x(t)$ that I observe at those time points, and would like to infer the discrete state $y(t)$ that is most likely to correspond to $x(t)$. If it matters, I know the distributions $x(y_1),\cdots,x(y_n)$, and that there are only six possible hidden states, and have lots of annotated data to train the model on.
Were it that simple, I think a hidden Markov model and the Viterbi algorithm would do the trick. But I know that the output at time $t_i$ is affected not only by the hidden state at time $t_i$, but by the three previous hidden states, like so:
I left the diagram undirected, since the correlations between successive hidden states are not causal, but I could live with rightward arrows all around if I have to.
Anyway, does anyone know what such a model might be called, and/or if it's even plausible to try to learn the parameters for it? It seems simple enough that I feel like someone has got to have done it before, but I haven't been able to locate anything. I think it's basically a linear-chain conditional random field, but those extra edges make me feel lost and confused.
EDIT:
If it's any easier to parse/think about, here's an example where the current observed state depends on the past two, rather than four, hidden states:
This is sort of the minimal extension of a linear chain CRF, but I still can't find an example of something like it. Any help?