0
$\begingroup$

Assuming we want to estimate the proportion of success and there are two stages. We will move to stage 2 only when the number of success is greater than equal to a threshold (say 5) in stage 1. And we want to use stage 1 and 2's information to estimate the proportion of success.

If we observe 3 successes out of 30 in stage 1 and 14 successes out of 100 in stage 2, can we simply say the estimated proportion is 17/130=?

I suspect there would be bias upward or downward as we include the stage 1's result knowing its success.

Any suggestions to address this bias and estimate the proportion?

Thanks in advance.

$\endgroup$
  • $\begingroup$ Seems garbled, maybe with typos. Please clarify: Exactly what is Stage 1? Under what circumstances do we move to Stage 2? If we move to Stage 2, exactly what is it? If we have only Stage 1, how do we estimate P(Success); if we have both stages, how do we estimate P(Success). $\endgroup$ – BruceET Nov 8 '18 at 0:37
  • $\begingroup$ Sorry for the typo. In stage 1, for samples 30 if we observe greater or equal to 5 success then move to stage 2 where we will collect larger sample 100. $\endgroup$ – Sue Nov 8 '18 at 0:42
0
$\begingroup$

If I understand the situation, I think you are right that there will be bias, and for the reason you give. If you start off with an estimate that is 'large' you try again, possibly substituting a smaller estimate based on results from more trials.

Simulation with Phases 1 & 2: I simulated the experiment a million times for each of three success probabilities $p = 1/10, 1/6, 3/10,$ respectively. The event cond occurs if we go on to Stage 2. The estimate ext is the average of results based on 30 trials if only Stage 1 is used, and the average of results based on 130 trials if both stages are used. [Notice that if $p = 1/6,$ then the average count at Phase 1 is $5.]$

set.seed(1118);  m = 10^6; p = .1
x1 = rbinom(m, 30, p);  x2 = rbinom(m, 100, p)
cond = (x1 >= 5)
est = x1/30;  est[cond] = (x1[cond]+x2[cond])/130
mean(est);  mean(cond)
[1] 0.08819467  # est seems < 0.1
[1] 0.175508    # aprx P(Go to Stage 2)

p = 1/6
x1 = rbinom(m, 30, p);  x2 = rbinom(m, 100, p)
cond = (x1 >= 5)
est = x1/30;  est[cond] = (x1[cond]+x2[cond])/130
mean(est);  mean(cond)
[1] 0.1461873   # est seems < 1/6
[1] 0.57612     # aprx P(Go to Stage 2)

p =.4
x1 = rbinom(m, 30, p);  x2 = rbinom(m, 100, p)
cond = (x1 >= 5)
est = x1/30;  est[cond] = (x1[cond]+x2[cond])/130
mean(est);  mean(cond)
[1] 0.3996667   # est seems near 0.4
[1] 0.998444    # aprx P(Go to State 2)

Straightforward simulation with 130 trials: If you just go ahead and use $n = 130$ trials from the start (with no Phase 1 or Phase 2 contingencies), you get very good estimates with a million iterations:

mean(rbinom(10^6, 130, .1)/130)
[1] 0.1000389  # aprx .1
mean(rbinom(10^6, 130, 1/6)/130)
[1] 0.1666442  # aprx 1/6
mean(rbinom(10^6, 130, .4)/130)
[1] 0.3999704  # aprx .4

Toward a general analytic solution: To find a general analytic solution for any $p,$ you need to start by figuring out such things as: (a) the probability of going on to Stage 2, (b) the expected value of the estimate $E(\hat p_1)$ for Stage 1 only, (c) the estimate $E(\hat p_1 | \text{Stop at Phase 1})$, and (d) the estimate $E(\hat p_b)$ if both stages are used.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.