# What happens if the observations are connected in a hidden Markov model (HMM)?

Suppose that we have an HMM with hidden variables $$X_t$$ and observed variables $$Y_t$$. Why do we always assume $$p(Y_t|X_t)$$? What happens if we have $$p(Y_t|X_t, Y_{t-1})$$? Is it because that wouldn't change the inference results on $$X_t$$ whether or not $$Y_t$$ depends on $$Y_{t-1}$$? Or is it a necessary assumption for the inference and learning algorithms to work correctly?

An example of an HMM is predicting the weather (hidden variable) based on the type of clothes that someone wears (observed). Here, the observations are not necessarily independent conditioned on the weather. For example, we might know that always $$Y(t+1) \neq Y(t)$$.

(I've added the red arrows to this figure from wikipedia: https://commons.wikimedia.org/wiki/File:Hmm_temporal_bayesian_net.svg#/media/File:Hmm_temporal_bayesian_net.svg)

One way to handle more dependency in the HMM model is to consider directly that the couple $$(X,Y)$$ is a Markov chain. Then the forward backward procedure can be extended and inference can be made to get an estimate of the hidden states. See for example Signal and Image Segmentation Using Pairwise Markov Chains by Stephane Derrode.