2
$\begingroup$

I am working with years data and I want to see how likely it is for something to have happened for the first time in, say, 1997 given other dates as well. Let me explain.

  1. I have something (a lab observation of a specific molecule) that has happened 10 times (in possibly different years) and I have all those years. x <- c(1997, 1998, 1998, 1998, 2000, 2005, 2006, 2006, 2010, 2019) Here I see the first occurrence is in 1997 and I wonder how likely is it that it happened then (so in a way, was that expected).

  2. I am interested in the first occurrence only (so 1997).

  3. I have a distribution (a much much larger pool of years, ~120k) from which I can resample my 10 occurence dates. These ~120k observations are actually all lab observations of the same type as mine (so they use similar techniques but different molecules). [This sampling distribution is very negatively skewed and between 1960 and 2020.]

  4. I want to know what's the probability of really having the first occurrence in 1997, so I thought I'd do this:

    • I resample from 3) 1000 times (10 years each time) and record the earliest year of occurence every time. So now I have a sample of size 1000 with the first years. I call this sample F. [* I'm not sure I can assume normality for the sample F, given the skewness of the initial sampling distribution, so I decided I need another method. This is why I went for the bootstrap.]

    • What I did next is resample from F with replacement and every time calculated the rank-turned-probability (probability) for 1997. This gives me a "sample" for the probability of having the first observation in 1997. And then I thought I can take the mean of those probabilities as the estimate for "the probability of observing 1997".

However, I didn't find any explicit basis to do that. It kind of looks like bootstrapping but bootstrapping, as I've seen it, has only been used for estimating confidence intervals around a metric (but is this a metric?). Is there a resource I can consult to confirm my method or to adjust it in order to make it correct? Or is this the right approach at all?

Any guidance would be greatly appreciated.

$\endgroup$
4
  • $\begingroup$ Can you be a bit more clear on what you mean that an event should happen the first time in 1997? What is the ~120k observations for and how do they relate to the first vector you show us? $\endgroup$ – Guilherme Marthe Feb 10 at 15:27
  • $\begingroup$ @GuilhermeMarthe I edited the question, I hope it's more informative now. $\endgroup$ – Magi Feb 10 at 16:52
  • $\begingroup$ if the earliest time happened in 1997, then when you resample, the earliest time in the resampled dataset will have to be 1997 or later, right? If that is wrong, can you give more details about what you have and what you will do? $\endgroup$ – John L Feb 10 at 23:11
  • $\begingroup$ @JohnL , I've written in there that my resampling distribution has values between 1960 and 2020, so I don't just shuffle these 10 years but everytime I draw 10 years from the distribution. So it is likely that I'll have different first years. $\endgroup$ – Magi Feb 11 at 9:57
1
$\begingroup$

You don't need to bootstrap.

Assuming iid, suppose the probability of occurrence in a given year is equal to $p$. If the start year is 1960, the probability of the first year being 1997 is $\left ( 1-p \right )^{37}\times p$. That is the probability of it not occurring in 1960 x ... x the probability of it not occurring in 1996 x the probability of it occurring in 1997. Look into the geometric distribution. It seems that once it occurs, it's more likely to occur again; but if you think you can assume iid until the event happens, this method is optimal.

If you have multiple datasets, you can estimate $p$ by the average 1/(average time until first occurrence).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.