# Detecting change in p of a Bernoulli process

A machine outputs either a 0 or a 1 each second. We denote this output at time $$t$$ as $$b_t$$. The probability that it outputs 1 is $$p_t$$ at time $$t$$. How do we go about studying the change in $$p_t$$ in $$t$$? This problem depends on the window in which you restrict your analysis. If your window is just 1 second, the probability changes very often but if the window encompasses the whole set of data you have, then you can not talk of any change. But determining the window demands knowledge of how $$p_t$$ changes. I would like to be pointed to references relating to this problem. Thank you.

• This question is significantly different depending on whether $p$ is continuously changing, or if it is constant for a while and then changes suddenly, etc. Can you clarify? – Xiaomi Oct 9 '18 at 23:38
• I was thinking about the latter. But I am also interested in the former as well. Sounds like the former is harder? – ztyh Oct 10 '18 at 0:03
• The former is significantly harder since you're estimating a continuous time process. For the latter, you're effectively detecting change points which is a well studied problem. – Xiaomi Oct 10 '18 at 0:07

OK, so you're really thinking of a discrete time process.

Well, then you can maybe define p(t) to be piecewise constant, for example like p($$0:t_1$$) = 0.5, p($$t_1:t_2$$) = 0.2, p($$t_2:t_3$$) = 0.5....p($$t_{S-1}:t_S$$)=0.7, in $$S$$ pieces.

Then you can try something like reversible jump MCMC to sample different $$S$$ and different vaues for $$t_1$$, $$t_2$$ etc, ie sample different # of changepoints.

https://people.maths.bris.ac.uk/~mapjg/papers/RJMCMCBka.pdf

Peter J. Green's paper in section 4 has an example, # of coal mining disasters, and the RJMCMC finds changepoints in the historic data.

In your case, instead of using Poisson distribution for # of disasters each year, you can use a Bernoulli distribution.

I've written a paper that uses Bernoulli likelihood (in small time windows) to infer a changing $$\lambda(t)$$ (or p_t, in your notation) given spike observations here: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005596

The above paper inferred a latent, unobserved $$\lambda(t)$$ from observation using Gibbs sampling and state space modeling, using Polya-Gamma data augmentation to sample for Bernoulli variables. http://www.academia.edu/download/46693506/Bayesian_Inference_for_Logistic_Models_U20160621-17860-1f6j9b3.pdf

You could just as well say $$\lambda(t_{n-1}:t_n)$$ is piecewise constant over $$S$$ segments, and use RJMCMC explained in Green's paper.

Short answer: no, as you describe it, you'll only get the trivial answer $$p_t = 1$$ when $$b_t=1$$. You need some additional assumptions.

Now before I try to answer, can I just get a clarification here?

You said “This problem depends on the window in which you restrict your analysis.” “choose a window that encompasses the whole dataset”

You originally said window size = 1 second, and probability of $$b_t = 1$$ is $$p_t$$.

If $$p_t$$ were large, say 0.5, if you increased your window size to 2 seconds, you’ll have possible observations of [0, 1, 2] - with windows having 2 pulses (“11”) in them being observed roughly 1/4 of the time. This can no longer be described by a Bernoulli process.

So just to check, were you envisioning a scenario where your machine is something that operates in continuous time, and spits out a pulse every so often?

If that is the case, you can approximately describe this machine in discrete time using a Bernoulli process IF the probability per unit time of outputing a pulse $$\lambda(t) \ll \frac{1}{\Delta t}$$, where $$\Delta t$$ is the discrete time window size.

———————————

The following is under the assumption that the above is the correct understanding of what you ask.

What you’re then asking is, can you infer $$\lambda(t)$$ given some data?

Well, you’ll need to make some additional assumptions about how quickly $$\lambda(t)$$ is allowed to change in time. Otherwise the trivial answer is that $$\lambda(t)$$ is highly concentrated around observed pulse times, akin to $$p_t = 1$$ where $$b_t = 1$$ and $$p_t = 0$$ where $$b_t = 0$$.

Using Gaussian kernels to estimate http://176.32.89.45/~hideaki/res/pdf/shimazaki_jcns10.pdf

If you had several realizations of this process, each of which is $$T$$ seconds long, and you assume that $$\lambda(t)$$ is shared between the processes, a simple way would just be to align them and make a histogram, and use that as an estimator for $$\lambda(0:T)$$, $$\hat \lambda(0:T)$$, and is used in neuroscience to estimate firing rates of a neuron triggered around some stimulus or behavioral event.
• I was thinking of discrete time. each $b_t$ is bernoulli with parameter $p_t$, where $t$ is only on the integers. The continuous stuff is interesting too though, thanks. – ztyh Oct 10 '18 at 0:11