1
$\begingroup$

I recently started trying to learn Bayesian analysis, but I'm not yet confident that I understand the concepts, so I did the following example and wrote out my understanding of the concepts. I would appreciate it if people could please take the time to review it and and provide feedback as to whether my understanding is correct.


I have performed two experiments. In the first experiment, I had $173/323$ successes. In the second experiment, I had $125/198$ successes.

I modelled the first experiment as the binomial random variable $X_1$ with parameters $n_1$ and $p_1$. I modelled the second experiment as the binomial random variable $X_2$ with parameters $n_2$ and $p_2$. I hold $n_1$ and $n_2$ fixed.

I'm trying to get a Bayesian estimate for each of $p_1$ and $p_2$ and find the standard deviation of their posterior distributions.

I know that the family of beta distributions is the conjugate prior for the binomial distribution, so I use that as the prior for the parameters $p_1$ and $p_2$. The hyper-parameters were chosen to be $\alpha = 0.5$ and $\beta = 0.5$ (Jeffrey's prior).

If my understanding is correct, at this point, one is now ready to run simulations of these random variables.

To illustrate, in Stan, my model is as follows:

y1 ~ binomial(n1, p1);
y2 ~ binomial(n2, p2);
p1 ~ beta(alpha, beta);
p2 ~ beta(alpha, beta); 

My understanding is that the "posterior distribution" is the distribution of $p_1, p_2$, right? The point of what we're doing here -- in Bayesian analysis and simulation -- is to find the (posterior) distribution of some unknown values (in this case, $p_1, p_2$), using the prior distribution (in this case, the beta distribution), which reflects our beliefs about the posterior distribution before running the simulation?

  1. Is my understanding of this correct?

So, after running the simulation, the standard deviation of the posterior distributions of $p_1$ and $p_2$ would be the standard deviation of the distribution that was generated by the simulation?

  1. Is my understanding here correct?

After I ran this simulation, I got that $p_1$ has an approximate mean of $0.54$, with standard deviation of approximately $0.03$, and $p_2$ has an approximate mean of $0.63$, with standard deviation of approximately $0.03$ (but different from the standard deviation of $p_1$).

  1. Does this sound correct? And, if so, would I be correct in saying that these mean and standard deviation values are the mean and standard deviation values of the posterior distributions for $p1$ and $p_2$?
$\endgroup$
2
$\begingroup$

Your understanding is generally correct, but I would like to make some observations that you may want to use.

Beginning with your quote

I have performed two experiments. In the first experiment, I had 173/323 successes. In the second experiment, I had 125/198 successes.

Is this the flip of a single "coin" or two different "coins." If it is two separate coins and there is no relationship between them, then you are correct. If there are two coins and they covary, then you are not correct. If there is one coin, then you are partially correct.

I am going to assume from your language that they are not going to covary as I would assume you would specify a linkage between them. However, if it is one coin, then the posterior density of the first experiment should be the prior density of the second experiment and not Jeffreys' prior as you should never use a vague prior when you have clear information.

As to your quote

I know that the family of beta distributions is the conjugate prior for the binomial distribution, so I use that as the prior for the parameters $p_1$ and $p_2$.

Why are you using the beta distribution. Conjugacy is nothing special. It only speeds calculation up and eliminates your need to perform simulations. If you are using the Jeffreys prior because you are missing information and it is invariant under transformation then your choice makes sense. It may not otherwise. I suggest you plot the three beta distributions of $\{(0,0),(.5,.5),(1,1)\}$ and ask if any of them fit your beliefs the best. If you are not performing a transformation of variables, then there is no advantage to using the Jeffreys prior.

The $(0,0)$ will give you the same results as the Frequentist solution and it will allow for double-headed and double-tailed coins. Jeffreys' prior does not allow for double-sided coins. The $(1,1)$ variant is what most people think of when they think of a distribution being uninformative. Note that when you graph them only $(1,1)$ grants equal probability to all options except the boundaries which have zero mass. Jeffreys' prior gives infinite mass to the boundaries and it minimizes in the center. Do you mean for that to be the case?

My understanding is that the "posterior distribution" is the distribution of $p_1$, $p_2$, right?

Their joint distribution is the posterior if there is one experiment for each parameter and not two experiments for one parameter. You could model the two parameters as two distinct distributions or as a joint distribution. However, because you applied the Jeffreys prior for the one dimensional case to each one, they should be reported separately.

Your exact mean for the first posterior is 0.535494 and your exact standard deviation of your beliefs regarding the first parameter is 0.027665. Your exact mean for the second posterior is 0.630653 and your exact standard deviation of your beliefs regarding the second parameter is 0.0341269.

While it is correct to say that they are the mean and standard deviation of your two posteriors, it may be better to say it is the mean and standard deviation of your posterior beliefs about the location of the parameter. When you say it that way, it sounds strange and I wanted to emphasize the strangeness to get you to think in terms of reporting the entirety of the two distributions rather than just the point summaries.

Point summaries do not play the same role in Bayesian statistics as in Frequentist ones. The sample mean and the sample standard deviation are used to construct statistical tests in Frequentist methodologies. They are incidental knowledge, a nice summary in Bayesian methods.

If $p_1$ is thought of as a population mean, then the mean you calculate is the mean estimate of the population mean and the standard deviation is the standard deviation of the mean estimate of the population mean.

Your simulation estimates of the mean and the standard deviation are approximations of these exact measures. If the simulation is good enough then the answer is yes. You can always run bad simulations. If you did them properly and the software should assure that, then yes they are reasonable approximations.

EDIT Let us assume there is no linkage at all between the experiments, then in that case, you would construct two posterior distributions and they would share no information. If you had traps in both locations and they are distant enough to not impact each other, then if green lizards lived in one area and there were a variety of other animals in the traps that are non-green lizards then that would be your trapping rate. If, in the other location, you had counted the rate of lizards and animals that are non-red lizards then you would perform two experiments with two posteriors.

$\endgroup$
  • $\begingroup$ Thanks for the excellence response, Dave. I have one question regarding "Their joint distribution is the posterior if there is one experiment for each parameter and not two experiments for one parameter." I said two experiments in the post, but I think this was incorrect. Let's say that this was a single experiment to catch lizards and record the number of red v green lizards. The lizards are either red or green. One trap caught 173 red lizards out of 323, and the second trap in a different location caught 125 red lizards out of 198. Would this be a case of two experiments for one parameter? $\endgroup$ – The Pointer Apr 4 '18 at 16:58
  • $\begingroup$ And what exactly is meant by "two experiments for one parameter"? In this context, does that mean that $p_1$ and $p_2$ are actually our attempts to estimate a population parameter $p$? $\endgroup$ – The Pointer Apr 4 '18 at 17:01
  • $\begingroup$ @ThePointer No in the case of a single coin, $p_1=p_2=p$. $\endgroup$ – Xi'an Apr 4 '18 at 18:16
  • $\begingroup$ @Xi'an In my lizard example, is there a single "coin"? $\endgroup$ – The Pointer Apr 4 '18 at 18:59
  • 1
    $\begingroup$ Yes, there is an unknown proportion $p$ of red lizards, which is observed in two successive experiments. Without further complications, both samples can be merged into a single Binomial $B(n_1+n_2,p)$. We have a related illustration in Chapter 5 of our book Bayesian Essentials. $\endgroup$ – Xi'an Apr 4 '18 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.