Timeline for How to tell the probability of failure if there were no failures?
Current License: CC BY-SA 3.0
26 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 13, 2017 at 12:44 | history | edited | CommunityBot |
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
|
|
| Jan 25, 2015 at 21:22 | comment | added | gung - Reinstate Monica | Thanks for the reference, @DWin. I find the rule of 3 often handy in simulation work & rare diseases. | |
| Jan 25, 2015 at 21:19 | comment | added | DWin | Yay for the "Rule of three". I fist saw it many years ago in a short note to the "Journal of the American Medical Association" jama.jamanetwork.com/article.aspx?articleid=385438 | |
| Jan 25, 2015 at 15:16 | history | edited | gung - Reinstate Monica | CC BY-SA 3.0 |
added 18 characters in body
|
| Jan 23, 2015 at 16:44 | history | edited | amoeba | CC BY-SA 3.0 |
fixed a typo
|
| Jan 23, 2015 at 16:38 | history | edited | gung - Reinstate Monica | CC BY-SA 3.0 |
added 349 characters in body
|
| S Jan 22, 2015 at 23:58 | history | suggested | D.W. | CC BY-SA 3.0 |
Clean up the answer so it stands on its own and reads better for a first-time viewer, and to avoid "Update: ..." (we dont' need "Update: ..." on this site, as we have revision history for that).
|
| Jan 22, 2015 at 23:57 | review | Suggested edits | |||
| S Jan 22, 2015 at 23:58 | |||||
| Jan 22, 2015 at 22:17 | comment | added | whuber♦ | @gung OK, comments don't give enough space to be sufficiently clear. I therefore posted an analysis based on (a variant of) this approach. It is not intended as a criticism of your answer (nor of Yair Daon's), both of which I like (and upvoted), but only as another approach worthy of consideration. | |
| Jan 22, 2015 at 21:10 | comment | added | gung - Reinstate Monica | @whuber, I'm not sure I follow that. What is $u()$ here? If it operates over the joint distribution of $(X,Y)$, we would still need estimates of the governing parameter $p$ for the 2 marginals. I liked amoeba's suggestion, & included it, but it is only correct contingent on that particular prior being the right prior. I am definitely more comfortable just using the upper 95% CL (which I actually do use on occasion). | |
| Jan 22, 2015 at 21:02 | comment | added | whuber♦ | The Bayes approach certainly is easier in this case! However, you can solve the problem as I framed it without having to fudge anything. E.g., to construct an upper prediction limit procedure for $Y\sim\text{Binom}(m,p)$, find a function $u:\{0,1,\ldots,n\}\to \{0,1,\ldots,m\}$ such that $\Pr(u(X)\ge Y)\ge 1-\alpha$ no matter what value $p$ has (and, hopefully, this probability is close to $1-\alpha$ in any case). The probability here refers to the joint distribution of $(X,Y)$ (i.e., before $X$ was observed). Since the OP observed $X=0$, the UPL is $u(0)$ with $n=100000$ and $m=10000$. | |
| Jan 22, 2015 at 21:01 | history | edited | gung - Reinstate Monica | CC BY-SA 3.0 |
added 194 characters in body
|
| Jan 22, 2015 at 20:55 | comment | added | gung - Reinstate Monica | @whuber, you're right. "Sure" in particular was a poor choice of words. This was only meant as a quick & dirty upper bound. To get an optimal upper bound I would presumably have to integrate over all possible non-0 counts weighted by the likelihood for each corresponding possible $\hat p$. That strikes me as more than the OP may really need, & I'm not actually sure how to do it outside of a Bayesian context (I upvoted Yair's answer, b/c it does use that approach). I will edit the "sure", though. | |
| Jan 22, 2015 at 17:27 | comment | added | whuber♦ | Your edit is good progress (+1). However, it raises issues of interpretation. We are not "sure" the chance is not more than $26\%$ because we are not completely certain of the true underlying chance. We do not have an "upper bound" on $p$, but only an upper confidence limit. When you give a prediction for a future event, you need to (a) estimate it and (b) provide bounds on it. Look at it like this: give us bounds on $Y$ when $X\sim\text{Binomial}(n,p)$, $Y\sim\text{Binomial}(m,p)$ independently, conditional on $X=0$. Those bounds are a prediction interval for $Y$ based on $X$. | |
| Jan 22, 2015 at 16:49 | comment | added | amoeba | Ah, cool, now it fits perfectly! Just a short note to explain Laplace's rule of succession. In a Bayesian framework, we start with a uniform prior $Beta(1,1)$ on the fail probability $p$. After observing $0$ failures out of $n$ trials, we update the prior and get $Beta(1+0, 1+n)$ as a posterior. It has expected value $p=1/(n+2)$, which is precisely Laplace's rule of succession. So the expected number of fails in the next $m$ trials is $m/(n+2)$ and the probability of getting at least one failure is $1-(1-1/(n+2))^m \approx m/(n+2)$, which in this case immediately gives $\approx 10\%$. | |
| Jan 22, 2015 at 14:11 | history | edited | gung - Reinstate Monica | CC BY-SA 3.0 |
added 74 characters in body
|
| Jan 22, 2015 at 14:02 | comment | added | gung - Reinstate Monica |
@amoeba, thanks for the tip. That was an error in my code. pbinom() gives you the probability up & including q; I should have used pbinom(0, ...). The predicted probabilities for 1+ are higher; the previous values were the probabilities for >1.
|
|
| Jan 22, 2015 at 13:59 | history | edited | gung - Reinstate Monica | CC BY-SA 3.0 |
deleted 2 characters in body
|
| Jan 22, 2015 at 13:06 | comment | added | Yair Daon | @amoeba as I mentioned, I took a uniform prior over the failure probability. I believe that a different prior would have lead to considerably different results. | |
| Jan 22, 2015 at 2:36 | history | edited | gung - Reinstate Monica | CC BY-SA 3.0 |
added 1 character in body
|
| Jan 21, 2015 at 21:44 | history | edited | gung - Reinstate Monica | CC BY-SA 3.0 |
added 995 characters in body
|
| Jan 21, 2015 at 19:29 | comment | added | whuber♦ | Gung, this is a good approach. But what is your answer to the question, which asks for a prediction limit that one of the next 10,000 products fails? Conceptually that's not quite the same thing as a confidence limit for the true failure rate (although obviously the two are closely connected), and numerically the answers are very different. | |
| Jan 21, 2015 at 19:25 | comment | added | whuber♦ | @amoeba This rule of 3 is a 95% one-sided confidence limit. Assume the failure count has a Binomial$(n,p)$ distribution. Then the chance of seeing no failures is $(1-p)^n$. To make that greater than $5\%$, solve $(1-p)^n\ge 0.05$ for $p$. Using $\log(1-p)\approx -p$ for small $p$, the solution is $p\le -\log(0.05)/n$. Since $0.05=1/20\approx e^3$, we obtain $p\le 3/n$. That's the "rule of 3." It's worth knowing because now you know how to vary the "3" if you want to adjust the confidence level and you also can invert it to find the minimum $n$ needed to detect a rate of $p$ or greater. | |
| Jan 21, 2015 at 19:15 | comment | added | gung - Reinstate Monica | @amoeba, thanks. I wasn't familiar w/ that. From Wikipedia I gather that it is an attempt to estimate $p$ (which "still requires a proof", although it seems quite reasonable). The rule of 3, OTOH, attempts to estimate the upper bound of the 95% CI. | |
| Jan 21, 2015 at 19:01 | comment | added | amoeba | +1. I have not heard about the "rule of 3" before. I wonder if there is any connection between the rule of 3 and "Laplace's rule of succession"? According to the latter (if I apply it correctly), probability of failure can be estimated as $1/(N+2)$. | |
| Jan 21, 2015 at 18:50 | history | answered | gung - Reinstate Monica | CC BY-SA 3.0 |