Developing a Bayesian model in R (rephrased)

Question

I wanted to rephrase an earlier question to try and be less vague. I'm essentially looking for help understanding basic Bayesian inference.

Let's say I want to predict the transfer rate of a high school in its first year of existence. To do this I feel like the ideal set up would be looking at it from a Bayesian perspective.

Since I don't have any historical information from the school, I want to take the transfer rates from other schools in the district (labeled rates).

rates <- c(.08, .11, .02, .04, .03, .06, .05, .02, .06)

I believe that these rates would be my prior. The issue is I'm not sure what my next step would be. Without data from the school, I'm not sure where the likelihood and posterior distribution would come from.

Any guidance or help would be appreciated. I feel like I'm looking at the question wrong, which if I am, feel free to say.

Answer 1

Apparently, whatever this "transfer rate of a high school" thing is, it seems to be a result of a binomial process and you are trying to find a model for each students "chance of a transfer", which I'll call $\theta$. Now the KISS approach would be to assume an identical $\theta$ in every school. A more sensible way would be to assume, that each school has a different $\theta$, taken out of a common distribution and the most sensible approach to assume, that each student has a different probability, which we cannot measure.

mef has written about the more sensible approach, named a hierarchical model and how you really need to learn more before you approach it. If this is just a self-study introductional example as to how Bayes works (please add self-study tag in that case), we can go the KISS way and assume, every student in the world has the same chance $\theta$.

Now it comes in handy, that there is a conjugate prior to the binomial distribution in the beta distribution. First find a prior distribution, i. e. what do you know about $\theta$ before you look at data. Let's say, you do know, that $\theta$ is not going to be 100% and else nothing. The least informative prior to state that would be a beta distribution with $a=1$ and $b=2$.

Updating: For each student without a transfer, increase $a$ by one and for each with a transfer, increase $b$ by one:

transfers <- c(8, 12, 2, 10, 19, 3, 42, 8, 19, 11)
totals <- c(111, 108, 79, 122, 130, 95, 201, 117, 122, 103)

nontransfers <- totals - transfers

Now through the wonder of conjugate priors, our posterior distribution of $\theta$ becomes this:

curve(dbeta(x, 1+sum(transfers), 2+sum(nontransfers)))
curve(dbeta(x, 1+sum(transfers), 2+sum(nontransfers)), xlim=c(0.05,0.15))

Thus, we have build an easy Bayes-Model to start with and explain the basics. The only downside is: It is far to easy and not acceptable. With the same beta-conjugate-magic we can easily see, that not all highschools have identical $\theta$. Just plot our $\theta$ distributions for all the highschools and you can see, that our model is far from acceptable:

curve(dbeta(x, 1+transfers[1], 2+nontransfers[1]), ylim=c(0,25), xlim=c(0,0.4))
for(i in 2:(length(transfers)))
    curve(dbeta(x, 1+transfers[i], 2+nontransfers[i]), add=TRUE)

But maybe even this is teaching something worthwhile.

Answer 2

Your idea is to learn about the transfer rate for a school for which you as yet have no data from a collection of related schools for which you do have data. This is a great idea and the Bayesian approach is the right way to go. In the Bayesian framework, your "willingness to learn" amounts to an assertion of prior dependence among the transfer rates for all the schools --- the ones for which you have data and the one for which you don't. A natural and convenient way to structure such a prior is through a hierarchical setup. The transfer rates (which are parameters) are modeled as independent and identically distributed with respect to a common distribution which itself has one or more unknown hyperparameters. The hyperparameters provide the channel through which the information flows.

However, you must first learn to walk before you can run. In other words, you need to understand how to learn about a single school's transfer rate before you can you can understand this more sophisticated problem. Take a single school with a transfer rate of .05 for example. Is that the "true" transfer rate or just an observation? Is that based on a school with 20 students with 1 transfer or 2000 students with 100 transfers. The amount of information is different in the two cases. This matters not only for learning the true transfer rate for this school but also for learning about the rate for another school.

Bayesian Data Analysis by Gelman (and others) is one place you can learn how to walk and then how to run. Chapter 5 (in the 3rd edition) is titled "Hierarchical Models." They have an example that has the same structure as your problem: Instead of transfer rates across schools, it's rat tumor rates across experiments. Their data includes the size of each experiment, which corresponds to the size of your schools. Although they do not emphasis the aspect of the problem you are interested in (i.e., "what have I learned about the next experiment/school?"), they do mention this in passing.

Other introductory textbooks discuss hierarchical models as well, such as Doing Bayesian Data Analysis by Kruschke. This book may be more suitable depending upon your background and interests. (Don't let the cover of the book put you off. I call the book "Doggy Bayes.")