2
$\begingroup$

I would like to model a process that is not really a random in nature - a sequence of events of a human behavior. The data would be so sparse that capturing two kinds of events for the same user in the right order (that would tell us a certain behavior) is very rare, and for majority of users we have to 'extrapolate'.

Can I apply Markov Model family here? It's meant to model random processes, while this process is not random, it will only look like it from the accessible data.

The initial idea was to represent a chain of states of a user as he gets closer to the desirable action.

To describe the idea briefly: There are events of different kinds user makes, where mentioned goal actions can be events as well that took place in the past. If the user has taken a certain number of events with a certain frequency for a period of time, he moves to the next state towards the final goal state. If there were no events there after, we roll him back to the previous state. What we want to achieve with this model is to define the right moment of time for intrusion to change human's behavior.

Is there a variation of Markov Model that would fit to this description? Maybe other type of model suits here?

Improve this question
$\endgroup$
1
  • $\begingroup$ the real problem is not the randomness but persistence and memory. it's hard to imagine that your projection of human's behavior in future only depends on the state of a human today, and not on what has been happening to him in the past, i.e. Markov assumption is questionable in its core. having said that sometimes it doesn't mean that Markovian approach will not work for anything involving humans. e.g. in stock markets the asset prices are driven by human behavior, which is persistent and obviously non-Markov yet the stock price returns were successfully modeled in Markovian frameworks. $\endgroup$
    – Aksakal
    Feb 28, 2018 at 21:41

1 Answer 1

Reset to default
1
$\begingroup$

The fact that you consider the process you observe / model to be "not really random" is not an obstacle. Probability theory is often used to described processes as stochastic even if there might be an underlying deterministic process in the case that this deterministic process is very complex / high dimensional and therefore cannot be modeled in detail. In fact, one of the schools in the interpretation of statistics (Bayesianism) holds that there is not necessarily such thing as "really random", and that probability theory deals with situations where we have insufficient information to reach definite conclusions. Therefore I don't see a problem in using a stochastic process model in your case.

The question whether a Markov model is appropriate is more complex. Markov chains are the first thing that comes to mind when dealing with transitions between discrete states, and human behavior in a certain conceptualization fits that bill. However, a Markov process also possesses the property that it is only the current state that has a (stochastic) influence on the next state, but none of the previous states does (the so-called Markov property). If this is not the case, but the dependence only goes back over a fixed finite number of previous states, one can consider a Markov chain of $n$th order, where $n$ is the number of previous states that exert an influence. If you don't have a good theoretical argument that your process possesses the Markov property, you can test for it or you can estimate the order, but this may necessitate a large amount of data to work properly.

For testing the Markov property, see e.g. Kullback, Kupperman, Ku (1962), "Tests for Contingency Tables and Markov Chains", Technometrics 4(4), 573-608, http://www.jstor.org/stable/1266291.

For estimating the Markov order, see e.g. Csiszár and Shields (2000), "The consistency of the BIC Markov order estimator", The Annals of Statistics 28(6), 1601-1619, http://www.jstor.org/stable/2673999.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks a lot. It definitely brings me closer to the solution. I was thinking, that a current state will be (partially) defined by a set of observed human actions. But the state itself is related to another, target action. So, there is no direct connection between observable action that define the state and final action. That is why I though I would create a model with certain rules of going from one state to another. Something is missing here. How do I specify a Markov model in this case? What do I need to define? $\endgroup$
    – mel
    Sep 10, 2015 at 1:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.