6
$\begingroup$

I'd like to use Bayes' Theorem on data obtained through a small random sample, and I want to use Agresti-Coull (or any other alternative technique) to know how big the uncertainty is.

Here is Bayes' Theorem:

$P(A|B) = \frac{P(B|A)\cdot P(A)}{P(B)}$

Now, all the data I have on this system is obtained from small random samples, so there's a large uncertainty involved with all three variables, $P(B|A)$, $P(A)$ and $P(B)$.

I've been using Agresti-Coull to obtain both the value and the uncertainty for each of these three variables. (I represent the number+-uncertainty as a ufloat object using the uncertainties package.)

But using Agresti-Coull three times separately for these three variables is a problem; They are dependent on each other. So I've been getting impossible results. For example, if you let $P(B)$'s uncertainty pull it downward, and the respective uncertainties of $P(B|A)$ and $P(A)$ pull them upwards, you get a total probability bigger than one.

Is there a way to do Agresti-Coull-style approximation on the whole Bayes expression instead of doing it on the three pieces separately?

$\endgroup$
7
  • 1
    $\begingroup$ As I understand the Agresti-Coull procedure is used to generate approximate confidence intervals for the binomial proportion confidence interval. I am not sure I see the relationship to Bayes theorem. Could you provide some context and your goals within the context to understand how the bayes arises in the context of the Agresti-Coull method? $\endgroup$
    – user28
    Oct 18, 2010 at 18:55
  • $\begingroup$ @cool-RR: How exactly are you estimating those three probabilities? $\endgroup$
    – whuber
    Oct 18, 2010 at 19:50
  • $\begingroup$ @whuber: For $P(A)$ I check how many of the messages in my finite sample have $A$, and divide that by the size of the sample. For $P(B|A)$ I check how many of the messages that have $A$ also have $B$, and divide that by the number of messages that have $A$. etc. $\endgroup$
    – Ram Rachum
    Oct 18, 2010 at 21:01
  • 2
    $\begingroup$ @cool-RR: why don't you just estimate P(A|B) directly, then, as the fraction of B samples that are also A? $\endgroup$
    – whuber
    Oct 18, 2010 at 21:03
  • 1
    $\begingroup$ @cool-RR: Just a small technical remark, to make things precise: the factory function ufloat() creates a UFloat object. Case does matter, here (the two names were chosen to be close to each other on purpose). :) $\endgroup$ Oct 28, 2010 at 8:18

4 Answers 4

5
$\begingroup$

When applying the formula for P(B|A) for Agresti-Coull, it seems important to me to use, for the denominator (ñ), a number with uncertainty. The formula ñ=P(A)*N+4 (where N is the size of your sample) gives you this number, after you calculate P(A) with an uncertainty. With the uncertainties package, this would be:

# Calculation of P(A):
P_A = ufloat(…, …)
# Calculation of P(B|A):
P_B_A = …/(P_A*N+4)

Thus, P(B|A) is automatically correlated to P(A).

Furthermore, you must make sure that you feed ufloat() with standard deviations. This mean using a particular z_{1-alpha/2} value, in the Agresti-Coull formula.

Hope this helps!

$\endgroup$
1
  • 1
    $\begingroup$ +1 thanks for pointing out how to do this with the uncertainties package. $\endgroup$
    – ars
    Oct 30, 2010 at 2:48
3
$\begingroup$

Error propagation won't work in the way handled by the uncertainties package. As you note, they're dependent, so you have to take the covariances into account.

You can obtain the variance of your distribution P(B|A) using the Delta Method and use that to obtain a confidence interval.

With Bayesian inference, you might find it simpler to use a credible interval. The following slides do a good job of explaining how to obtain this:

$\endgroup$
1
1
$\begingroup$

Brown, Cai, and DasGupta, AS, 2002

Brown, Cai, and DasGupta, Stat Sci, 2001

I don't know if I understand you correctly, but in my knowledge the above two papers are the most cited ones recently when it comes to binomial proportions' CI and estimation.

Sorry if this is not what you wanted.

$\endgroup$
0
$\begingroup$

This is a potential answer to the title of the original question and not necessarily the body of the question...

Looking at the Agresti confidence interval measurement, to my eyes it bears a resemblance to a Bayesian estimator.

https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Agresti-Coull_Interval

Looking at p = 1/n * ( X + 1/2 * z^2), I suspect the 1/2 could be considered Bayesian prior knowledge.

Let’s say for example, that one conducts a congressional approval poll and historically those polls have an approval rating of 0.35. Using a Bayesian train of thought perhaps one could use that information as prior knowledge and feed it in by using 0.35 instead of the 1/2 coefficient?

I suspect one could do so successfully.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.