I would like to detect user anomalies or mal-behavior on a web site. For each user I monitor the web browser used, IP (and thus ISP & geo-location) of the user as well as users' activities on the site (there are ~10 types of activities). I thought to construct a hidden Markov model for this type, but all the examples I see on the net use only a single predictor, while here we have at least three families: (1) web browser information; (2) information derived from user IP and (3) user activity on the site. How are these predictors combined into a single Markov model?

Also, should the training set data be pre-labeled as normal/abusive?

PS: Why do I insist using Markov models instead of other classification methods? The truth is that I have a pretty successful neural network classifier working. I'm exploring Markov (or HMM) models mainly as an exercise, but also as a potential way to deal with the labeling problem: confirming non-normal behavior is hard, not all user activities that I believe are "unusual" are indeed so, and moreover, there might be abusive activities that may be unaware of.

  • 1
    $\begingroup$ Can't figure out how to combine what different types of information? $\endgroup$ – jerad Dec 14 '12 at 22:29
  • $\begingroup$ Why not use a classification algorithm instead of an HMM, which assumes (in practice) some evolution of states? (This is a real, not a rhetorical, question.) $\endgroup$ – jbowman Dec 14 '12 at 23:29
  • $\begingroup$ Perhaps Mixtures of HMMs would be appropriate, where one HMM is normal and the other is abnormal browsing behavior. You could then classify user behavior by inferring which HMM best describes a user's behavior. Google Automatic categorization of web pages and user clustering with mixtures of hidden Markov models for a very similar application of Mixtures of HMMs. $\endgroup$ – jerad Dec 15 '12 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.