21
$\begingroup$

I read quite a bit of hidden Markov models and was able to code a pretty basic version of it myself.

But there are two main ways I seem to learn. One is to read and implement it into code (which is done) and the second is to understand how it applies under different situations (so I can better understand how it relates to problems I might be working on). All the examples I have done so far have involved either some kind of DNA prediction or coin tossing.

I'm wondering if there are any resources to get other Markov problems (language doesn't matter but hopefully with the answers as well so I can know if I'm right or wrong)?

|cite|improve this question
$\endgroup$
  • $\begingroup$ I was advised to cross post this from stackoverflow.com/questions/8661941/… $\endgroup$ – Lostsoul Dec 29 '11 at 7:03
  • $\begingroup$ Could you be a little more specific in terms of "code a pretty basic version"? Did you simulate from a Hidden Markov process, or did you code the Viterbi, forward, or Baum–Welch algorithms? (The last three would be used to compute the most-likely corresponding sequence of states, the probability of the sequence of observations, or the starting probabilities, transition function, and observation function of a hidden Markov model, respectively.) $\endgroup$ – Wayne Dec 29 '11 at 18:13
  • $\begingroup$ Hi Wayne, I basically coded a version of this page(the spreadsheet) for baum-welch: cs.jhu.edu/~jason/papers/#tnlp02 and basically implemented the code for the viterbi wiki page and followed a few basic tutorials on hidden markov models. This might sound stupid but I wanted to see other types of problems I could try to solve so I can gain a better understanding of what markov models are capable of. $\endgroup$ – Lostsoul Dec 29 '11 at 19:49
  • 1
    $\begingroup$ I don't want to spend weeks working on it, but for example a case study of someone using markov models in a non-coin toss or weather prediction way might help me understand the range of problems it can solve better. i'm basically looking to build a better understanding by testing what markov models can do. $\endgroup$ – Lostsoul Dec 29 '11 at 19:51
  • $\begingroup$ I think HMM also have very important applications in Finance(interest rates) and Economics(GDP). $\endgroup$ – An old man in the sea. Sep 23 '14 at 22:06
8
$\begingroup$

I've used HMM in a demand / inventory level estimation scenario, where we had goods being purchased from many stores that might or might not be out of inventory of the goods. The sequence of daily demands for these items thus contained zeroes that were legitimate zero demand days and also zeroes that were because the store was out of stock. You would think you'd know whether the store was out of stock from the inventory level, but errors in inventory records propagate and it is not at all uncommon to find a store that thinks it has a positive number of items on hand, but actually has none; the hidden state is, more or less, whether the store actually has any inventory, and the signal is the (daily demand, nominal inventory level). No references for this work, though; we were not supposed to publish the results for competitive reasons.

Edit: I'll add that this is especially important because, with zero demands, the store's nominal on hand inventory doesn't ever decrease and cross an order point, triggering an order for more inventory - therefore, a zero on hand state due to erroneous inventory records doesn't get fixed for a long time, until somebody notices something is wrong or a cycle count occurs, which may be many months after the problem has started.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ I believe this is known as a zero-inflation problem, and they're fairly widespread. You need one model that models the "excess zeroes" (when the reading is zero because there cannot be any reading, as opposed to a legitimate reading of zero), then a second-level model that models the rest. For example, the number of customers in a bank: sometimes there actually are none, other times the bank is closed so there can't be any. Or the speed of a car: sometimes it's sitting still with a driver in it, other times it's parked. Etc. $\endgroup$ – Wayne Dec 30 '11 at 19:26
  • $\begingroup$ True enough, from the point of view of the demand signal. The other part of the problem is identifying the binary "inventory = 0 | inventory record > 0" hidden state, which was actually more important for the customer. $\endgroup$ – jbowman Dec 30 '11 at 19:28
  • $\begingroup$ I should also point out that the "inflated zeroes" are not i.i.d. over time - there are runs where all the zeroes are "extra" and runs where none of them are, hence the need for the HMM with the state indicating which is happening at each observation. $\endgroup$ – jbowman Dec 30 '11 at 21:11
6
$\begingroup$

I pretty much experienced the same thing and didn't find much beyond the weather. Areas that come to mind include: speech recognition, change point detection, tagging parts of speech in text, aligning overlapping items/text, and recognizing sign language.

One example I found and did some exploration of was in Section 8 of this introduction, which is one of the references for HMM's in Wikipedia. (It's actually pretty fun: your analysis discovers that there are vowels and consonants.) This also introduces you to working with a text corpus, which is useful.

(If you want to play with generation with HMMs, you could train on Shakespeare text and then generate faux-Shakespeare.)

|cite|improve this answer
$\endgroup$
3
$\begingroup$

Most speech recognition software uses Hidden Markov Models. You can experiment with natural language processing if you want to get a feel for HMM applications.

Here's a good source: Probabilistic Graphical Models, by Koller and Friedman.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks, Carlos. Great book, I started reading it a while back but didn't finish it. Got it to learn about machine learning and graph theory, but I'll go back and look for questions related to markov models. I'll also take a look at natural language processing(I have never worked with it before) $\endgroup$ – Lostsoul Dec 29 '11 at 19:52
3
$\begingroup$

Hidden markov models are very useful in monitoring HIV. HIV enters the blood stream and looks for the immune response cells. It then sits on the protein content of the cell and gets into the core of the cell and changes the DNA content of the cell and starts proliferation of virions until it burst out of the cells. All these stages are unobservable and called latent. An ideal example for hidden markovian modelling.

|cite|improve this answer
$\endgroup$
  • 2
    $\begingroup$ So in what way exactly do hidden Markov models help to monitor HIV? Do clinicians use HMMs to diagnose HIV? Do researchers use them to better understand mechanisms of the disease or create anti-HIV drugs and therapies? Any references would be very helpful. $\endgroup$ – Leo Dec 12 '12 at 14:29
1
$\begingroup$

For me, very nice application of HMM is chord identification in musical composition. See for example this lecture.

|cite|improve this answer
$\endgroup$
0
$\begingroup$

Markov models may be useful in analyzing the interactions of a user with a website - For Example on Amazon.com where figuring out what series of interactions lead to a checkout to give recommendations in the future.

A fun example showing the use of Markov Model is the following-

http://freakonometrics.blog.free.fr/index.php?post/2011/12/20/Basic-on-Markov-Chain-(for-parents)

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ Not HIDDEN Markov models here though - huh? $\endgroup$ – B_Miner Jan 4 '12 at 0:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.