How does one formalize a prior probability distribution? Are there rules of thumb or tips one should use?

Question

While I like to think I have good grasp of the concept of prior information in Bayesian statistical analysis and decision making, I often have trouble wrapping my head around its application. I have in mind a couple of situations that exemplify my struggles, and I feel that they are not properly addressed in the Bayesian statistical textbooks I've read so far:

Let's say I ran a survey a few years ago that says that 68% of people would be interested in purchasing an ACME product. I decide to run the survey again. While I'll be using the same sample size as last time (say, n=400), people's opinions have likely changed since then. However, if I use as a prior with a beta distribution in which 272 out of 400 respondents answered "yes", I'd be giving equal weight to the survey I ran a few years back and the one I'd be running now. Is there a rule of thumb to establish the greater uncertainty I'd like to place on the prior by virtue of that data being a few years old? I understand I can just reduce the prior from 272/400 to, say, 136/200, but this feels extremely arbitrary, and I wonder if there's some form of justification, perhaps in the literature, to back up the choice of prior.

For another example, let's say we are about to run a clinical trial. Before launching the trial, we run some secondary research which we could use as prior information, including expert opinions, results from previous clinical trials (of varying relevance), other basic scientific facts, etc. How does one go about combining that spectrum of information (some of which is non-quantitative in nature) to a prior probability distribution? Is it just a case of making a decision on which family to pick and make it diffuse enough to ensure it gets overwhelmed by the data, or is there a lot of work done to establish a fairly informative prior distribution?

Answer 1

Your idea to treat your prior information of 272 successes in 400 attempts does have fairly solid Bayesian justification.

The problem you are dealing with, as you recognized, is that of estimating a success probability $\theta$ of a Bernoulli experiment. The Beta distribution is the corresponding "conjugate prior". Such conjugate priors enjoy the "fictitious sample interpretation":

The Beta prior is $$ \pi(\theta)=\frac{\Gamma(\alpha_0+\beta_0)}{\Gamma(\alpha_0)\Gamma(\beta_0)}\theta^{\alpha_0-1}(1-\theta)^{\beta_0-1} $$ This can be interpreted as the information contained in a sample of size $\underline{n}=\alpha_0+\beta_0-2$ (loosely so, as $\underline{n}$ need not be integer of course) with $\alpha_0-1$ successes: $$ \pi(\theta)=\frac{\Gamma(\alpha_0+\beta_0)}{\Gamma(\alpha_0)\Gamma(\beta_0)}\theta^{\alpha_0-1}(1-\theta)^{\underline{n}-(\alpha_0-1)} $$ Hence, if you take $\alpha_0+\beta_0-2=400$ and $\alpha_0-1=272$, this corresponds to prior parameters $\alpha_0=273$ and $\beta_0=129$. "Halving" the sample would lead to prior parameters $\alpha_0=137$ and $\beta_0=65$. Now, recall that the prior mean and prior variance of the beta distribution are given by $$ \mu=\frac{\alpha}{\alpha+\beta}\qquad\text{and}\qquad\sigma^2=\frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)} $$ Halving the sample keeps the prior mean (almost) where it is:

alpha01 <- 273
beta01 <- 129
(mean01 <- alpha01/(alpha01+beta01))

alpha02 <- 137
beta02 <- 65
(mean02 <- alpha02/(alpha02+beta02))

but increases the prior variance from

(priorvariance01 <- (alpha01*beta01)/((alpha01+beta01)^2*(alpha01+beta01+1)))
[1] 0.0005407484

to

(priorvariance02 <- (alpha02*beta02)/((alpha02+beta02)^2*(alpha02+beta02+1)))
[1] 0.001075066

as desired.