Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 6 down vote favorite
1

Suppose I have 5 possible events which either happened or did not happen in the preceding 10 time periods. How do I figure out how probable any event is in the 11th period?

    t11 t10 t9  t8  t7  t6  t5  t4  t3  t2  t1
A       1   1   1   1   1   1   1   1   1   1
B       0   0   1   1   0   1   1   1   1   1
C       0   0   0   0   0   0   0   0   1   1
D       1   0   1   1   0   1   0   1   0   1
E       0   1   1   1   1   1   0   0   0   0
share|improve this question
Well, yes, I get that. But how would that work with this actual data? – user3671 Nov 22 '12 at 13:01
Initial priors are 50:50, with updating at every time period. Rows are independent, columns are obviously not. Sorry about the confusion. – user3671 Nov 22 '12 at 13:38
I mean the probability at each time period depends on the calculated probability of the preceding time period, the probability at t0 being .5. – user3671 Nov 22 '12 at 13:43
Can you be more specific about what you need help understanding? Are you familiar with Beta-Binomial models? – jerad Nov 22 '12 at 17:48

1 Answer

up vote 10 down vote accepted

Assume the outcome of each trial is a coin flip that only depends on some constant, unknown bias $\theta$ which you are trying to infer so you can predict the next outcome after seeing some data. Assume that $\theta$ itself was drawn from some prior distribution, which we'll assume to be a Beta distribution with paramaters $a,b$, depicted graphically below

Beta prior

Then the generative model for your data can be written as $$ \theta \sim Beta(a,b) $$ $$ X \sim Binomial(n,\theta) $$ where $X$ is number of successes (or 1's in your example) out of $n$ trials.

The first distribution is known as your prior and the second is your likelihood. The Beta distribution is conjugate to the Binomial distribution, which means your posterior is still a Beta distribution. I assume you're familiar with Bayes' rule at least in theory so I'll just explain practically how to update your belief distribution in this model and make predictions about upcoming trials.

The predictive distribution in the case of the Beta-Binomial model is simply the expectation (the mean) of your belief about $\theta$, which for $Beta(a,b)$ is $\frac{a}{a+b}$. So for example if you have no reason to assume a priori that any value b/t 0 and 1 is more likely than any other, you could set $a=b=1$ so so that your belief was totally uniform(see plot). Then your prediction is $\frac{1}{1+1} = 0.5$.

Say that you observe 10 trials with 8 successes and 2 failures. The posterior distribution is then $Beta(a+8,b+2)$. Notice that the paramaters $a,b$ of your $Beta(a,b)$ prior can be interpreted as "psuedo-observations", where $a$ is the number heads and $b$ the number of tails that you have in effect hallucinated, since they're treated the same as actual observations are in your posterior belief.

So you can easily calculate the predicted outcome for your examples above, but you have to assume some parameter values $a$ and $b$ for your prior. Then your prediction is simply $\frac{a+x}{a+b+N}$, where $x$ is the number of successes observed and $N$ is total number of trials.

share|improve this answer
1  
Jerad... That makes a whole lot of sense now. You should write a book. Many thanks. – user3671 Nov 23 '12 at 5:23

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.