Questions on issues with using Frequentist and Bayesian approach for the same test One quick stats question, if I use Binomial Cumulative Distribution Function to get a sample size n for desired confidence level and tolerable error. Then we pick a sample of sample size n and find k failures in the sample. Is it ok for us to use Jeffreys Interval instead of the exact method (clopper Pearson interval) for getting the upper limit of the estimated failure rate in the population? 
My concern is that the sample size is obtained using Frequentist while the Jeffreys Interval is Bayesian approach. So can we still draw conclusion on the upper limit of the estimated failure rate with a conference level? 
 A: Bayesian interval estimation. Let $\theta$ be P(Success) and the prior distribution be $\theta \sim
\mathsf{Beta}(.5,.5).$ Then if you observe $x$ successes in $n$ trials, the likelihood is proportional to $\theta^x(1-\theta)^{n-x},$ so that the
posterior distribution of $\theta$ conditional on the data is
$\mathsf{Beta}(.5+x,\, .5+ n-x).$ Then a 95% Bayesian posterior
probability interval uses quantiles .025 and .975 of the posterior
beta distribution. (To get a one-sided interval with an upper bound, use
quantile 0.95.)
If you are willing to overlook the difference in interpretation between a
Bayesian posterior (or credible) interval and a frequentist confidence interval,
you might use the Bayesian interval based on the Jeffreys prior as a confidence interval. With $x = 3$ successes in $n = 100$ trials, the one sided 95% Jeffreys 
interval would give the upper bound $0.068.$ The corresponding two sided 95% interval is
$(0.0085, 0.0779).$  [Computation in R.]
qbeta(.95, 3.5, 97.5) 
[1] 0.06875679
qbeta(c(.025, .975), 3.5, 97.5)
[1] 0.008520283 0.077887569

Frequentist Confidence intervals. In a frequentist context with a normal approximation, the Wald 95% CI is
of the form $\hat \theta \pm 1.96 \sqrt{\frac{\hat \theta(1-\hat\theta)}{n}}.$
where $\hat \theta = x/n.$
If $\theta$ is far from $1/2$ and $n$ is relatively small, the Wald interval (originally proposed as an asymptotic interval) may not have the advertised
95% coverage probability: first, because the normal approximation may not be accurate; second, because the standard error may not be well approximated by using $\hat \theta$ instead of $\theta.$ In the cases $x = 0$ and $x =n,$ the Wald interval degenerates to a point. Agresti ("plus-four"), Wilson, and Clopper-Pearson intervals are substantially better, mitigating the second difficulty, but not the first.
For comparison, the two-sided 95% Agresti CI (formula here) based on 3 successes in 100 trials
is $(0.0070, 0.0892).$
n = 104;  p = 5/104; pm = c(-1,1)
p + pm*1.96*sqrt(p*(1-p)/n)
[1]  0.006961131 0.089192715


True coverage probabilities of frequentist CIs vary as $\theta = P(\text{Success})$ varies. Because of the discreteness of the binomial distribution, the variation can be surprisingly large. For $n = 100,$ the actual coverage probability of the "95%" Wald interval is below 95% for 'most' values of $\theta.$ The performance is
sufficiently bad that its use is now deprecated.

Agresti 95% CIs generally have close to the promised coverage probability (and sometimes a little better
near $\theta = 0$ and $1.)$ While other intervals may be a bit more accurate, the Agresti style of CI is widely used because its formula is easy to remember and use.

When regarded as a frequentist CI, the Jeffries interval is reasonably accurate for most values of $\theta.$ (Regarded as a Bayesian posterior probability interval, the Jeffries interval always contains 95% probability according to the posterior distribution. So from a Bayesian point of view, if you believe the Prior and trust the integrity of the data, then you believe
the interval. Remember that in a Bayesian framework $\theta$ is a random variable, not a fixed, unknown value.)

