# Help me understand Bayesian updating

Suppose I have 5 possible events which either happened or did not happen in the preceding 10 time periods. How do I figure out how probable any event is in the 11th period?

    t11 t10 t9  t8  t7  t6  t5  t4  t3  t2  t1
A       1   1   1   1   1   1   1   1   1   1
B       0   0   1   1   0   1   1   1   1   1
C       0   0   0   0   0   0   0   0   1   1
D       1   0   1   1   0   1   0   1   0   1
E       0   1   1   1   1   1   0   0   0   0

• Well, yes, I get that. But how would that work with this actual data? Nov 22, 2012 at 13:01
• Initial priors are 50:50, with updating at every time period. Rows are independent, columns are obviously not. Sorry about the confusion. Nov 22, 2012 at 13:38
• I mean the probability at each time period depends on the calculated probability of the preceding time period, the probability at t0 being .5. Nov 22, 2012 at 13:43
• Can you be more specific about what you need help understanding? Are you familiar with Beta-Binomial models? Nov 22, 2012 at 17:48
• "Initial priors are 50:50" doesn't make any sense. Your priors are over the unknown biases, which aren't Bernoulli. Feb 24, 2015 at 2:55

Assume the outcome of each trial is a coin flip that only depends on some constant, unknown bias $\theta$ which you are trying to infer so you can predict the next outcome after seeing some data. Imagine that $\theta$ itself was drawn from some prior distribution, which we'll assume to be a Beta distribution with paramaters $a,b$, depicted graphically below

Then the generative model for your data can be written as $$\theta \sim Beta(a,b)$$ $$X \sim Binomial(n,\theta)$$ where $X$ is number of successes (or 1's in your example) out of $n$ trials.

The first distribution is known as your prior and the second is your likelihood. The Beta distribution is conjugate to the Binomial distribution, which means your posterior is still a Beta distribution. I assume you're familiar with Bayes' rule at least in theory so I'll just explain practically how to update your belief distribution in this model and make predictions about upcoming trials.

The predictive distribution in the case of the Beta-Binomial model is simply the expectation (the mean) of your belief about $\theta$, which for $Beta(a,b)$ is $\frac{a}{a+b}$. So for example if you have no reason to assume a priori that any value b/t 0 and 1 is more likely than any other, you could set $a=b=1$ so so that your belief was totally uniform(see plot). Then your prediction is $\frac{1}{1+1} = 0.5$.

Say that you observe 10 trials with 8 successes and 2 failures. The posterior distribution is then $Beta(a+8,b+2)$. Notice that the paramaters $a,b$ of your $Beta(a,b)$ prior can be interpreted as "psuedo-observations", where $a$ is the number heads and $b$ the number of tails that you have in effect hallucinated, since they're treated the same as actual observations are in your posterior belief.

So you can easily calculate the predicted outcome for your examples above, but you have to assume some parameter values $a$ and $b$ for your prior. Then your prediction is simply $\frac{a+x}{a+b+N}$, where $x$ is the number of successes observed and $N$ is total number of trials.

• Jerad... That makes a whole lot of sense now. You should write a book. Many thanks. Nov 23, 2012 at 5:23
• You should set the prior $a=b=\frac12$ in accordance with the Jeffreys prior. If you don't want to do that, then you must state your parametrization of the bias as being interpreted as a probability as another assumption. Feb 24, 2015 at 2:54