Designing an experiment: Geometric or Bernoulli data

Question

I have some process that succeeds or fails with probability $p$. I can do distinct simulations to estimate $p$:

1. Run $N$ simulations of a single process, record $N$ samples of a $\text{Bernoulli}(p)$, do GLM with binomial family to estimate $\log(p/(1-p))$.
2. Run $N$ simulations in which I continue running the process until a success, record $N$ samples of a $\text{Geometric}(p)$, do GLM with gamma family to estimate $p$.

What is the theoretical difference? Which way is better? I thought (1) is nicer but then I can't get $p$, only $\text{logit}(p)$, right? (see answer below).

UPDATE: I do regression because there is a simulation parameter t and I want to determine if p depends on t and how. Also, the above Ns are not the same, because each simulation in the geometric design is roughly equivalent to 1/p simulations in the Bernoulli design. But this is part of the question - is it better to make each trial on its own (Bernoulli) or to make sequences of trials? How does this design affect the power/significance/other statistical measures of both estimation of p and estimation of the regression parameters?

Answer 1

There is not much difference between the methods other than that the number of trials in method 1 is more predictable than in method 2. Consider the following example:

Say we tried 5 simulations using method 2 and the results are:

2, 2, 1, 4, 1

So our estimate of p = 1/mean = .5

This also means that we have run 10 trials:

0, 1
0, 1
1
0, 0, 0, 1
1

We can see this as a method 1 simulation and see 5 successes out of 10 trials, which corresponds to a estimate of p of .5

This is no coincidense. For method 2 our estimate of $p$ is 1/mean, which is $\frac{N} {\sum_i k_i}$ (whereby $k_i$ are the number of trials till success). We also know that this represents a method 1 simulation with $\sum_i k_i$ trials and $N$ successes, so our estimate will again be $\frac{N} {\sum_i k_i}$. So there is no real difference between method 1 and method 2 other than that the cost of method 1 is more predictable than the cost of method 2. As long as you compare like with like, that is, compare a method 2 simulation with a method 1 simulation with $\sum_i k_i$ trials, you will find no difference in the results with respect to the point estimates. As you use different variance functions with gamma and logit families you might get slight differences in standard errors. In Stata I would inspect this like so:

clear all
set obs 100
gen t = _n <= 50
gen p = .1 + .3*t
gen u1 = runiform() < p
local i = 1
sum u1, meanonly
local stop = r(mean) == 1

while `stop' == 0 {
    local j = `i'
    local i = `i' + 1
    gen u`i' = runiform() < p if u`j' == 0
    sum u`i', meanonly
    local stop = r(mean) == 1
}

egen k = rownonmiss(u*)

glm k t, family(gamma) link(power -1)
margins, over(t) predict(xb)

gen id = _n
reshape long u, i(id) j(iter)
glm u t, family(binomial) link(logit)
margins, over(t)

The relevant outputs are:

for the gamma model:

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           t |
          0  |   .0988142   .0112884     8.75   0.000     .0766895     .120939
          1  |    .462963    .052888     8.75   0.000     .3593043    .5666216
------------------------------------------------------------------------------

for the logit model:

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           t |
          0  |   .0988142   .0132661     7.45   0.000     .0728132    .1248152
          1  |    .462963   .0479803     9.65   0.000     .3689232    .5570027
------------------------------------------------------------------------------

So the predicted $p$ are exactly the same and there are slight differences in the standard error.

Answer 2

I thought (1) is nicer but then I can't get p, only logit(p), right?

Not so.

$\hat p = \frac{1}{1+\exp(-\hat\eta)}$ where $\eta = \text{logit}(p)$

That is, you just transform back.