8
$\begingroup$

So I've been compiling a list of fictional (anime/manga) characters which meet a certain criteria (http://www.gwern.net/hafu#list) from a universe of all anime/manga characters since 1963 (which is of unknown total size - but very large!), and I've been wondering how I could estimate how complete my list is at any point.

My final goal is to slice the character data and look for trends by year or decade; getting an idea of how large my sample actually is may help me estimate systematic bias. If I've gotten a large fraction of estimated characters, then I can hope that any decade trends may be real and not just a case of 'looking under the lamppost'.

What sort of techniques would be useful here? I've looked, and "capture-recapture" seems like an answer (especially reading Predicting total number of bugs based on number of bugs revealed by each tester )

But I'm not clear whether it really applies here. What would a "recapture" be, in my scenario? Would I have to keep track of every Google search or Google Alerts result or list of hafu characters I run into, and write down whether each entry was already present in my master list? Are there assumptions in capture-recapture that aren't satisfied here?

UPDATE: since no one replied yea or nay, I went ahead and did it:

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Can I just say, this is a fantastic problem. $\endgroup$ – Kyle. Nov 16 '12 at 1:46
  • $\begingroup$ I must say, I like your reaction better than that of the people who just stare and stare until I drop the subject. :) $\endgroup$ – gwern Nov 16 '12 at 4:57
5
$\begingroup$

Capture-recapture relies on random samples on both occasions. You somehow need to make a random sample from the unknown pool of characters, count them, then make another, totally independent sample, and count the overlap. In biology, capture-recapture works because you expect that marked fish mingle with others in the closed volume to the extent that two months later, the marked fish have distributed themselves in the entire volume of the lake, so by taking another random measurement, you can get an independent sample. Thus randomness of the two samples is comprised of (i) taking the samples of fish in randomly selected portions of the lake; (ii) relying on natural biological mixing processes to re-distribute the marked fish around.

Implementing capture-recapture in social sciences (which is where your little project falls into) is very difficult, as some people/characters are systematically easier to capture into the sample than others. Sometimes, network samples are used to estimate the sizes of unknown hard to reach populations, but they rely on some sort of social dynamics to move the sample on.

Among other works that tried to look into how complete a given collection/list may be, I can recall David Banks' work on how complete Wikipedia is and Brad Efron's and Ron Thisted's empirical Bayes work on how many words Shakespeare knew. So using capture-recapture for literary work is about 35 years old -- sorry to break the bad news to you ;).

I think if you produce a solid research, this would be a very appropriate publication for Significance joint magazine of the Royal Statistical Society and the American Statistical Association.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yes, the randomness assumption is questionable. Rcapture claims to be able to model heterogeneity in capture probabilities (which I think is the obvious way the randomness assumption would break down in this case), I hoped that it would be able to either show that it was not a problem or correct for it - but the full set of models took too much RAM to run! As far as I can tell, my data sources are all independent of each other since they don't cite each other and often didn't overlap much at all, but this doesn't prove that they weren't drawing from a common pool ranked by popularity... $\endgroup$ – gwern Feb 12 '13 at 22:52
  • 1
    $\begingroup$ So I just don't know. I'll be sure to look to look at those 2 papers; I was sure I couldn't be the first to apply the idea but didn't find those. I'll think about writing something for Significance, since the articles I see look cool. $\endgroup$ – gwern Feb 12 '13 at 22:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.