8
$\begingroup$

Assuming n testers were independently testing the same application for a given period. Each tester found a given set of bugs (Some of the bugs were detected by more than one tester).

For example:

Tester 1 found bugs {1,2,3,4,5} Tester 2 found bugs {3,5,6,7} Tester 3 found bugs {1,3,5,8,9,10}

(The serial number is assigned to each bug once it is detected by some tester, according to the detection order, nevertheless, the testers are unaware of each other, and of the assigned serial numbers.)

Assuming all bugs have equal probability to be detected.

(but without assuming that all testers are equally talented)

Can I calculate the probability of having x undetected bugs? How?

Improve this question
$\endgroup$
5
  • $\begingroup$ You have told us some testers may be better than others. Are some bugs easier to find than others? If so, there may be no upper limit to the number of bugs, but if each bug is equally easy to find (for a tester of a given level of skill) then there be ways of answering your question. $\endgroup$
    – Henry
    Commented Oct 1, 2011 at 11:14
  • $\begingroup$ @Henry: For each tester - each bug is equally easy to find. $\endgroup$
    – Lior Kogan
    Commented Oct 1, 2011 at 11:21
  • $\begingroup$ Are we assuming here that you also know the number of total bugs (i.e., these are bugs that were introduced)? $\endgroup$
    – mako
    Commented Oct 1, 2011 at 14:22
  • 1
    $\begingroup$ @Benjamin Mako Hill: Of course I do not know the number of bugs - this is what the question is all about. $\endgroup$
    – Lior Kogan
    Commented Oct 1, 2011 at 15:08
  • 1
    $\begingroup$ Interesting; similar to the problem of determining the number of species: draw n balls from an urn with unknown composition and determine the number of types of balls in the urn. $\endgroup$
    – Karl
    Commented Oct 1, 2011 at 15:56

1 Answer 1

Reset to default
6
$\begingroup$

This is a capture-recapture question, in this case with more than two visits, briefly discussed by Wikipedia and in more detail by the R vignette on the package Rcapture

Your assumptions of

  1. different testers having different skills ($t$)
  2. each bug being equally likely to be caught (not $h$)
  3. the probability of a bug being caught not being affected by whether it was previously caught (not $b$)

suggests an $M_t$ model. Using the R code

library(Rcapture)
bugscaught <- matrix(c(1,1,1,1,1,0,0,0,0,0,
                       0,0,1,0,1,1,1,0,0,0,
                       1,0,1,0,1,0,0,1,1,1),ncol=3)
closedp(bugscaught)

gives the following

Number of captured units: 10 

Abundance estimations and model fits:
              abundance   stderr  deviance  df     AIC
M0                 13.1      3.1     6.781   5  23.614
Mt                 12.9      3.0     6.128   3  26.961
Mh Chao            26.3     20.9     2.472   4  21.305
Mh Poisson2       115.9    237.3     2.472   4  21.305
Mh Darroch        696.0   2253.0     2.472   4  21.305
Mh Gamma3.5      4565.3  19853.4     2.472   4  21.305
Mth Chao           25.6     20.0     1.708   2  24.541
Mth Poisson2      113.6    232.5     1.708   2  24.541
Mth Darroch       699.7   2266.0     1.708   2  24.541
Mth Gamma3.5     4714.8  20515.0     1.708   2  24.541
Mb                 16.7     13.7     6.526   4  25.359
Mbh                 1.0     13.6     5.751   3  26.584

and for an $M_t$ model suggests a central estimate of 12.9 bugs with a standard deviation of 3, so I would suggest a range of from 10 to something like 20. If you want to dig deeper then the contents of closedp(bugscaught)$glm$Mt may have useful information.

As you can see, if you instead assume some bugs will be harder than others to find then depending on your model the central estimate could reach the thousands.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you Henry. I have doubts about this model, since I can't figure out how the estimator for having x bugs can have a normal pdf. In my example, I would expect the probability be 0 for any value less than 11. As suggested, I can enforce a range of 10 to 20, but is this the right way to go? $\endgroup$
    – Lior Kogan
    Commented Oct 2, 2011 at 11:26
  • $\begingroup$ @Lior: It doesn't have a normal distribution. It will closer in distribution to something like a geometric random variable such as $\Pr(B=b)=3^{b-10}/4^{b-9}$ for $b\ge 10$, though that has mean 13 and standard deviation about 3.46. $\endgroup$
    – Henry
    Commented Oct 2, 2011 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.