46
$\begingroup$

I'm just getting my feet wet in statistics so I'm sorry if this question does not make sense. I have used Markov models to predict hidden states (unfair casinos, dice rolls, etc.) and neural networks to study users clicks on a search engine. Both had hidden states that we were trying to figure out using observations.

To my understanding they both predict hidden states, so I'm wondering when would one use Markov models over neural networks? Are they just different approaches to similar problems?

(I'm interested in learning but I also have another motivation, I have a problem that I'm trying to solve using hidden Markov models but its driving me bonkers so I was interested in seeing if I can switch to using something else.)

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ You might want to look here: stats.stackexchange.com/questions/4498/… $\endgroup$
    – Nucular
    Commented Feb 16, 2012 at 16:26
  • $\begingroup$ Would you care to choose an answer, or further clarify what you're looking for? $\endgroup$
    – conjugateprior
    Commented Mar 9, 2014 at 12:32
  • $\begingroup$ HMMs in some simplistic way can be viewed as a subclass of the framework of dynamic Bayesian networks. $\endgroup$
    – Lerner Zhang
    Commented Mar 1, 2020 at 15:37

3 Answers 3

Reset to default
36
$\begingroup$

What is hidden and what is observed

The thing that is hidden in a hidden Markov model is the same as the thing that is hidden in a discrete mixture model, so for clarity, forget about the hidden state's dynamics and stick with a finite mixture model as an example. The 'state' in this model is the identity of the component that caused each observation. In this class of model such causes are never observed, so 'hidden cause' is translated statistically into the claim that the observed data have marginal dependencies which are removed when the source component is known. And the source components are estimated to be whatever makes this statistical relationship true.

The thing that is hidden in a feedforward multilayer neural network with sigmoid middle units is the states of those units, not the outputs which are the target of inference. When the output of the network is a classification, i.e., a probability distribution over possible output categories, these hidden units values define a space within which categories are separable. The trick in learning such a model is to make a hidden space (by adjusting the mapping out of the input units) within which the problem is linear. Consequently, non-linear decision boundaries are possible from the system as a whole.

Generative versus discriminative

The mixture model (and HMM) is a model of the data generating process, sometimes called a likelihood or 'forward model'. When coupled with some assumptions about the prior probabilities of each state you can infer a distribution over possible values of the hidden state using Bayes theorem (a generative approach). Note that, while called a 'prior', both the prior and the parameters in the likelihood are usually learned from data.

In contrast to the mixture model (and HMM) the neural network learns a posterior distribution over the output categories directly (a discriminative approach). This is possible because the output values were observed during estimation. And since they were observed, it is not necessary to construct a posterior distribution from a prior and a specific model for the likelihood such as a mixture. The posterior is learnt directly from data, which is more efficient and less model dependent.

Mix and match

To make things more confusing, these approaches can be mixed together, e.g. when mixture model (or HMM) state is sometimes actually observed. When that is true, and in some other circumstances not relevant here, it is possible to train discriminatively in an otherwise generative model. Similarly it is possible to replace the mixture model mapping of an HMM with a more flexible forward model, e.g., a neural network.

The questions

So it's not quite true that both models predict hidden state. HMMs can be used to predict hidden state, albeit only of the kind that the forward model is expecting. Neural networks can be used to predict a not yet observed state, e.g. future states for which predictors are available. This sort of state is not hidden in principle, it just hasn't been observed yet.

When would you use one rather than the other? Well, neural networks make rather awkward time series models in my experience. They also assume you have observed output. HMMs don't but you don't really have any control of what the hidden state actually is. Nevertheless they are proper time series models.

Improve this answer
$\endgroup$
3
  • 3
    $\begingroup$ +1 This is very good. By: Similarly it is possible to replace the mixture model mapping of an HMM with a more flexible forward model, e.g., a neural network., do you mean replacing the emission probabilities p(Observed|Hidden) with a feed forward NN? I have come across this in a few places but none gives an explanation; they just mention they have implemented it. I assume they replace the MLE step for learning emissions but cannot understand how. Do you know of any code or explanatory example? Any pointers welcome, thanks. $\endgroup$
    – Zhubarb
    Commented Sep 26, 2014 at 7:21
  • $\begingroup$ It seems to be less used that way around (some more ML person can correct me here). Tha said, estimating NN parameters mapping state vector to output conditional on knowing the state vector (or at least on knowing its expected value, as in an EM algorithm) is the same task as training with observed input and output data, so I guess there nothing special to say about that part. $\endgroup$
    – conjugateprior
    Commented Sep 26, 2014 at 8:40
  • $\begingroup$ The answer is good, except: Neural networks can be either discriminative (feed forward etc) or generative (restricted bolzmann machines etc). Also, recurrent neural networks like LSTM and Reservoir Computing models can model time series as well as HMM - and sometimes even better than HMM, especially in case of time series with strong nonlinear dynamics and long-time correlation. $\endgroup$
    – GuSuku
    Commented Nov 21, 2014 at 22:19
10
$\begingroup$

Hidden Markov Models can be used to generate a language, that is, list elements from a family of strings. For example, if you have a HMM that models a set of sequences, you would be able to generate members of this family, by listing sequences that would be fall into the group of sequences we are modelling.

Neural Networks, take an input from a high-dimensional space and simply map it to a lower dimensional space (the way that the Neural Networks map this input is based on the training, its topology and other factors). For example, you might take a 64-bit image of a number and map it to a true / false value that describes whether this number is 1 or 0.

Whilst both methods are able to (or can at least try to) discriminate whether an item is a member of a class or not, Neural Networks cannot generate a language as described above.

There are alternatives to Hidden Markov Models available, for example you might be able to use a more general Bayesian Network, a different topology or a Stochastic Context-Free Grammar (SCFG) if you believe that the problem lies within the HMMs lack of power to model your problem - that is, if you need an algorithm that is able to discriminate between more complex hypotheses and/or describe the behaviour of data that is much more complex.

Improve this answer
$\endgroup$
6
  • 6
    $\begingroup$ +1 For the second paragraph. I would like to point out that anyone who clearly understands all the elements of this answer probably wouldn't be asking the original question. It's probably not helpful to mention formal grammars to someone whose post starts with "I'm just getting my feet wet in stats..." The second paragraph here captures the essence of what the OP is asking. Instead of the first paragraph, you might say: an HMM models conditional dependencies of hidden states, where each state has a probability distribution over the observations. $\endgroup$
    – Phillip Cloud
    Commented Apr 6, 2012 at 17:30
  • 8
    $\begingroup$ This answer is simply wrong. The Neural Network here is assumed to be feedforward. This is only one class of neural networks. Recurrent models do not simply map single inputs down to a lower-dimensional representation, and they can generate language. See for example arxiv.org/abs/1308.0850 $\endgroup$
    – rd11
    Commented Mar 14, 2016 at 12:22
  • $\begingroup$ Generating a sequence (as in the linked paper) is not the same as generating a language. I suppose that you could apply them to discern between elements of a set and otherwise if you wish, however, a recurrent model can be envisaged as taking a single large input spanning over the individual inputs with recurrence and returning one large output. Not sure if the recurrent Neural Network can give you the outputs without any inputs. $\endgroup$
    – Andrew
    Commented Mar 14, 2016 at 20:17
  • $\begingroup$ Hmm. Can you give an example of something you think an HMM can generate, and that you believe can't be generated with an RNN? $\endgroup$
    – rd11
    Commented Mar 15, 2016 at 14:35
  • $\begingroup$ The example that comes to mind is the following: given an HMM you can obtain a sequence of elements that belong to the language that the HMM represents. For an RNN to do so, you need to add something over and above it (e.g. try the different inputs and mark an input as a member of a class or otherwise) - although in the case of RNNs you're probably looking at multiple inputs (one after the other) as representing a single "item". HMMs are more naturally suited for the purpose of generating a language. $\endgroup$
    – Andrew
    Commented Mar 15, 2016 at 20:05
7
$\begingroup$

The best answer to this question from what I have found is this: Is deep learning a Markov chain in disguise. This is exactly what I understood, but since there was already a discussion elsewhere in the Internet, I am putting the link here.

Markov chains model: $$p(x_1....x_n) = p(x_1)p(x_2 | x_1)p(x_3 | x_2) ...$$

RNNs attempt to model: $$p(x_1 .... x_n) = p(x_1)p(x_2 | x_1)p(x_3 | x_2, x_1)p(x_4 | x_3, x_2, x_1) ... $$

We can use a character sequence as the input instead of a single character. This way, we can capture the state better (depending on the context).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.