1
$\begingroup$

This question relates to the ".632 rule" asked about in an earlier question here.

My question is, assuming we have a process that approximates the random-sample-with-replacement for large (unknown) n contemplated by the rule, how can we measure our progress along the asymptotic curve toward $1/e?$

I had an inchoate idea, basically that we keep track of the frequency of previously noted marbles (so to speak) and use either the change in frequency over a given number of samples or use the absolute number ab initio as a measure of progress.

The bottom line is a pragmatic determination of a point of diminishing return. Roughly speaking, if I am a biologist sampling a pond's population of rotifers, how do I know when I have seen about all I am likely to see?

Thanks.

$\endgroup$
2
$\begingroup$

The .632 rule just comes from the fact that $$ \lim_{n \rightarrow \infty} 1-(1-\frac{1}{n})^n = 1-\frac{1}{e} \approx 0.632 $$

People have worked through the derivation of that rule in a few questions (e.g., here and here). In brief, it's a consequence of 1) sampling with replacement and 2) choosing a specific item during sampling is a Bernoulli trial.

For a finite number of samples, you could just plug your own value for $n$. However, its value approaches $1-\frac{1}{e}$ pretty quickly: It's within 5% for $n \ge 7$ and 1% for $n \ge 30$, so probably can't use that directly.

However, you're totally right that you can use similar logic to estimate the total number of specimens in a population. These studies are often called capture-recapture studies.

Suppose you have a pond of turtles. On day 1, you capture some of them and somehow mark them (e.g., band a bird or paint an X on the turtle's shell), keeping track of the total number of animals you saw and marked $(T_1)$. The next day/week/etc, you come back and capture more turtles, keeping track of the total number of turtles captured in this session $(T_2)$ and the number of marked turtles that were recaptured $(R)$. The Chapman estimator and its variance for the total number of turtles in the pond is then: $$ N=\frac{(T_1+1)(T_2+1)}{R+1}-1 \\ \textrm{ } \\ Var(N)=\frac{(T_1+1)(T_2+1)(T_1 - R)(T_2 - R)}{(R+1)(R+1)(R+2)}$$

This model makes some fairly strong assumptions, including:

  • Having exactly two capture sessions,
  • Animals are equally likely to be captured/recaptured,
  • A "closed" population where individuals individuals neither enter nor leave it.
  • Animals stay marked (the mark doesn't fade, fall off, etc)

and the estimates are somewhat biased. There are much more complicated models that include population dynamics, multiple rounds of recapturing, and so on.

Here's a worked example from Andrewartha

2,000 weevils were put into a box of wheat and allowed to disperse. In the first 'capture' session, 498 weevils were drawn, without replacement, from the box. A small dot was painted onto each captured animal. One week later, 110 weevils were drawn from the box. Twenty of the weevils were marked. Therefore, we'd estimate the total number of weevils as:

$$ N = \frac{(498+1)(110+1)}{20+1}-1 = 2,6366 $$

which isn't terribly far off. I'm not sure how you would do this exact experiment with rotifers, as they seem too small to mark (unless you could somehow dye or radiolabel them). Some animals (e.g., whales) have unique markings already, but I'd be surprised if rotifers do. If all else fails, I imagine the standard solution is to use a hemocytometer.

$\endgroup$
  • 1
    $\begingroup$ Incidentally, you can do even neater things if the individuals have serial numbers. Check out the story of the "German tank problem": theguardian.com/world/2006/jul/20/secondworldwar.tvandradio $\endgroup$ – Matt Krause May 26 '14 at 3:04
  • $\begingroup$ This answer is interesting and responsive. I like the simplicity even at the expense of strong assumptions. +1 $\endgroup$ – daniel May 26 '14 at 13:11
  • $\begingroup$ Thanks! There is an even simpler estimator ($N=\frac{T_1T_2}{R}$), but it's more biased and you seem like you can handle a little extra arithmetic :-) $\endgroup$ – Matt Krause May 28 '14 at 1:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.