I have two measurements from two different distributions. I know both of these distributions are binomial and I measure $k_1$ successes from $n_1$ trials for distribution 1 and $k_2$ successes from $n_2$ trials from distribution 2. From this, I estimate the probability parameter for each distribution as ${p_1} = \frac{k_1}{n_1}$ and ${p_2} = \frac{k_2}{n_2}$.

As well as having point estimates of $p_1$ and $p_2$, I'd like to quantify the uncertainty in some way. After reading this excellent post I'm convinced that what I want is a confidence interval.

To get a confidence interval for each of my estimates I can just use any of the methods on this page.

However, I also want to calculate different properties that are functions of $p_1$ and $p_2$, $f(p_1,p_2)$ and would like to get a confidence interval of the resulting quantities. This is where I get stuck. I can find some information on how to calculate a confidence interval for specific forms of $f$, e.g. if I know $f(p_1,p_2) = \frac{p_1}{p_2}$. But I can't find any way to calculate a confidence interval for general forms of $f$. What if I want a confidence interval on $f(p_1,p_2) = log(p_1^2+p_2^2)$? Or some other complicated form?

I can see how I could do this if I was using a bayesian credibility interval for $p_1$ and $p_2$. In this case I just randomly sample from the posterior for $p_1$ and $p_2$, and find limits on the distribution of $f(p_1,p_2)$ under this random sampling. This is what I will do if there is not a good alternative.

However, I prefer the interpretation of confidence intervals. Is there some numerical technique for calculating a confidence interval for $f(p_1,p_2)$? Or does it have to be worked out in detail for every particular form of $f$?

  • One general idea which just might work out, is to calculate numerically a profile likelihood function for the new parameter $f$, and then base a confidence interval on likelihood asymptotics. – kjetil b halvorsen Feb 18 at 17:24

This is a great question that will not yield a simple answer.

There is no general solution that provides the form of an arbitrary function $f$ of two or more random variables. You might be able to use the change-of-variable technique to compute the distribution function of your new random variable - see eg here or here. However, from there you will still need to calculate the CIs and again there is no general solution for this for an arbitrary distribution.

Having said that, if you're only concerned about getting reliable confidence limits and you're not concerned about the functional form or theoretical properties of your distribution, bootstrapping is a simple solution.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.