Suppose I have 5 possible events which either happened or did not happen in the preceding 10 time periods. How do I figure out how probable any event is in the 11th period?

    t11 t10 t9  t8  t7  t6  t5  t4  t3  t2  t1
A       1   1   1   1   1   1   1   1   1   1
B       0   0   1   1   0   1   1   1   1   1
C       0   0   0   0   0   0   0   0   1   1
D       1   0   1   1   0   1   0   1   0   1
E       0   1   1   1   1   1   0   0   0   0
  • $\begingroup$ Well, yes, I get that. But how would that work with this actual data? $\endgroup$
    – user3671
    Nov 22, 2012 at 13:01
  • $\begingroup$ Initial priors are 50:50, with updating at every time period. Rows are independent, columns are obviously not. Sorry about the confusion. $\endgroup$
    – user3671
    Nov 22, 2012 at 13:38
  • $\begingroup$ I mean the probability at each time period depends on the calculated probability of the preceding time period, the probability at t0 being .5. $\endgroup$
    – user3671
    Nov 22, 2012 at 13:43
  • $\begingroup$ Can you be more specific about what you need help understanding? Are you familiar with Beta-Binomial models? $\endgroup$
    – jerad
    Nov 22, 2012 at 17:48
  • $\begingroup$ "Initial priors are 50:50" doesn't make any sense. Your priors are over the unknown biases, which aren't Bernoulli. $\endgroup$
    – Neil G
    Feb 24, 2015 at 2:55

1 Answer 1


Assume the outcome of each trial is a coin flip that only depends on some constant, unknown bias $\theta$ which you are trying to infer so you can predict the next outcome after seeing some data. Imagine that $\theta$ itself was drawn from some prior distribution, which we'll assume to be a Beta distribution with paramaters $a,b$, depicted graphically below

Beta prior

Then the generative model for your data can be written as $$ \theta \sim Beta(a,b) $$ $$ X \sim Binomial(n,\theta) $$ where $X$ is number of successes (or 1's in your example) out of $n$ trials.

The first distribution is known as your prior and the second is your likelihood. The Beta distribution is conjugate to the Binomial distribution, which means your posterior is still a Beta distribution. I assume you're familiar with Bayes' rule at least in theory so I'll just explain practically how to update your belief distribution in this model and make predictions about upcoming trials.

The predictive distribution in the case of the Beta-Binomial model is simply the expectation (the mean) of your belief about $\theta$, which for $Beta(a,b)$ is $\frac{a}{a+b}$. So for example if you have no reason to assume a priori that any value b/t 0 and 1 is more likely than any other, you could set $a=b=1$ so so that your belief was totally uniform(see plot). Then your prediction is $\frac{1}{1+1} = 0.5$.

Say that you observe 10 trials with 8 successes and 2 failures. The posterior distribution is then $Beta(a+8,b+2)$. Notice that the paramaters $a,b$ of your $Beta(a,b)$ prior can be interpreted as "psuedo-observations", where $a$ is the number heads and $b$ the number of tails that you have in effect hallucinated, since they're treated the same as actual observations are in your posterior belief.

So you can easily calculate the predicted outcome for your examples above, but you have to assume some parameter values $a$ and $b$ for your prior. Then your prediction is simply $\frac{a+x}{a+b+N}$, where $x$ is the number of successes observed and $N$ is total number of trials.

  • 1
    $\begingroup$ Jerad... That makes a whole lot of sense now. You should write a book. Many thanks. $\endgroup$
    – user3671
    Nov 23, 2012 at 5:23
  • $\begingroup$ You should set the prior $a=b=\frac12$ in accordance with the Jeffreys prior. If you don't want to do that, then you must state your parametrization of the bias as being interpreted as a probability as another assumption. $\endgroup$
    – Neil G
    Feb 24, 2015 at 2:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.