# Jeffreys Prior vs. Empirical Bayesian analysis

I have a small data set, provided at the very end, where I have computed Jeffreys Prior to being a Beta(.5,.5) distribution. I then use this Jeffreys prior to report a 95% posterior credible set, assuming a binomial data set, which gives the proportion results as,

I then go ahead and use the actual proportions i.e. p1=12/(12+50) and find the mean and variance of those combined proportions to find my a and b values for my posterior distribution which is a beta(a+Y,b+n-Y),n=sample size,y=number of approvals, a and b are shape parameters. Using this distribution I now calculate the new 95% posterior credible set for each county which yields,

What the values mean here are the proportions we would expect (95% Confidence) to get the approval rates if we sampled each county.

What I'm struggling to see is what the pros and cons of using each method are. For the first method, we use Jeffreys prior to calculate the credible set and don't do anything with the actual proportion value. Where in the second method we use the actual proportions that are provided by the data set, also known as an Empirical Bayesian analysis.

Here's the data set and thanks for the input,

• What would your credible intervals be if County 1 had $0$ Approve and $62$ Disapprove? – Henry Dec 14 '19 at 19:12
• Well in that case, the intervals would be from 0 to 0.14 since the data still calculates the a and b values based on the remaining approvals of the other two counties. Which does make sense for 0 to be there since county 1 has 0 approval. – Warhawk1987 Dec 14 '19 at 19:24
• I'm not really sure what you've done in the second part for the empirical Bayes. Did you create a prior for each county using some sample of the data you have in the final table? Clarification would be appreciated. – Demetri Pananos Dec 14 '19 at 21:39
• @DemetriPananos, no the priors stayed the same for each county. The only difference is that the proportions were used to be able to get the empirical Bayes results, which ended up updating the priors to be a=5.88 and b=5.38 roughly for each county. – Warhawk1987 Dec 14 '19 at 21:47

What you are describing in the second portion is not empirical Bayesian analysis. An empirical prior is a prior collected from a parameter that you are not interested in, to estimate the parameter you are interested in.

For the Beta-Binomial, there are three standard uninformative distributions. The Haldane prior B(0,0) produces a maximum a posteriori estimator equal to the maximum likelihood estimator. It also allows for double-sided coins. The Jeffreys prior, B(.5,.5), produces results that are invariant to monotone transformations. It does violate the strong likelihood principle. The uniform distribution, B(1,1) distribution, has the intuitive property that it assigns an equal prior probability to all choices.

The empirical prior would use information in the sample that you are not interested, to improve the estimators of the parameters that you are interested in, such as the grand mean. One example of an empirical prior for this set would be B(182,263). Because the prior would so dominate the posterior and since it would not make sense to consider such a strong prior if the alternative is a Jeffreys prior, an approximately mean preserving prior would be B(4,6).

For your first example, there would be no changes, but with a B(4,6), you end up with slightly different credible sets (not confidence intervals).

The pro for using an empirical prior is it is a shrinkage prior in the same sense that Stein's method is. The con is that it may not be representative of any county. Imagine county one is in California, county two is in Montana, and county three is in the United Kingdom. The grand mean may add an illogical distortion to the data. It still provides valid shrinkage a la Stein which is a Frequentist tool, but it may offend against the underlying axioms such as Cox's axiomatic construction of probability which could be thought to undergird Bayesian thinking.

The pros for a Jeffreys' prior when there is no valid prior knowledge is that it is minimally informative and excludes the extrema of $$\pi\in\{0,1\}$$ for a true parameter. So observing $$\alpha=0$$ and $$\beta=50$$ does not imply $$\pi=0$$. The con of a Jeffreys prior is that it potentially offends against the strong likelihood principle. It could be seen to offend against the underlying axiomatic constructions, particularly, de Finetti's axiomatization or Savage's. After all, who actually would hold the belief that they saw half a head and half a tail on a coin toss unless you possibly meant that it could land on its side.

Both methods cross over into Frequentist thinking, which is not a bad thing, but you do want to realize that you are doing it and the limitations that such thinking may imply. There are three views on Frequentist ideas you can take.

You can hold the idea of "Heaven forbid that Frequentist ideas contaminate my analysis," or, "the prior could be arrived at without using a Frequentist idea so it is okay anyway," or finally, "why am I not using a Pearson-Neyman model and contaminating my results with Bayesian thinking." If you are worried about the information you are adding with your prior so much that you do not want to add anything or only add from the data, then why are you using a Bayesian method?