Imagine such a memory test for a mice. The mice performs an experiment $E$ with two possible issues $0$ (failure) and $1$ (success). If the mice gets $1$ it is "rewarded", if it gets $0$ it is "punished".

The experiment $E$ is repeated many times with the same mice, and we consider the stochastic process $(X_n)$ modelling the successive experiments. I wonder about the way to analyze such data. The mice is supposed to learn over time so we cannot assume independence and we cannot even assume a homogeneous Markovian transition.

We can assume inhomogenous Markov for simplicity. We are interested in the transition probabilities $p^n_{ij}=\Pr(X_{n+1}=j\mid X_n=i)$ ($i,j \in\{0,1\}$). Actually the two probabilities $p^n_{01}$ and $p^n_{11}$ determine the Markov step at time $n$.

With several mices it should be possible to estimate the $p^n_{ij}$. But with a moderate sample size we could assume a parametric constraint to have "power". Even with a large sample size it is interesting to assume a parametric constraint in order to see a "trend" over time. For instance we could assume a logit relation $\text{logit } p^n_{ij} = \alpha + \beta n$. At first glance it should be easy to estimate this models with maximum-likelihood.

My requests are:

  • Are there some statistical models of the same spirit in the literature ? I'm interested to get some references (not necessarily restricted to the Markov assumption).

  • I am also interested in getting some references --- if there are some --- about the more complex case of a $\{0,1\}$-valued continuous-time process $(X_t)$ modeling in addition the duration of the experiment $E$. More precisely, the mice can achieve success in a more a less rapid time, and the experiment is stopped after one minute if the mice has not yet succeed. For instance, we could assume for simplicity that the mouse always succed experiment $E$ after a time following an exponential distribution.

| cite | improve this question | | | | |
  • $\begingroup$ I would rather think about a hidden markov (or latent) process. The success probability develops randomly with time, and conditional on that, each learning experiment is independent. So search this site for logistic time series models? $\endgroup$ – kjetil b halvorsen Sep 4 '18 at 18:32

I know the question is old, but I was at the OCNS 2018 conference a few months ago. Daniel Wolpert presented work on this paper. I don't pretend to understand all of it, but I believe he modeled decisions as a stochastic process hitting a decision boundary. That is, to say,

Let $X_t\in \mathbb{R}$ represent the internal state of mouse mind. It is given two choices, represented by $\alpha, \beta\in \mathbb{R}$.

Let $T_\alpha = \inf(t|X_t>\alpha)$, $T_\beta = \inf(t|X_t<-\beta)$, and $T:= \min(T_\alpha, T_\beta)$.

If we can write $dX_t = \mu(t,X_t)dt+\sigma(t,X_t)dt$, then we can sometimes compute the distribution of decision time and which decision is made. I've only done the calculation for simple cases, and I don't remember the answer. However, a search of google/SE for "First hitting time of Brownian Motion" will reveal that when $\mu = 0$, $\sigma = 1$, and $\beta = \infty$, then $f_T(t) = \frac{|\alpha|}{t\sqrt{2\pi t}}\exp{(-\alpha^2/(2t)})$

To adapt this to your question, the functions $\mu$ and $\sigma$ would be $\mu_i$ and $\sigma_i$ as the mouse learns. $\mu_i$ could be something like the ratio of successes to total trials (like a Hebbian classifier), which would themselves be random variables.

| cite | improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.