2
$\begingroup$

Let's say I consider an event rare if it occurs no more than once in 90 days. Assuming everything is random and independent,

  1. If I see this event on day 3 of the observation, what is the probability that it's rare?
  2. If after that I see another event like this on day 80, what is the probability that this one is rare too?
Improve this question
$\endgroup$
5
  • $\begingroup$ I don't think this is answerable at present. Are you thinking of something like a Poisson process? If so, is it homogenous? Is day 0 some kind of natural starting point (say when the most event occurred? Also, are you thinking of this from a Bayesian point of view? (From the frequentist point of view, there isn't such a thing as a "probability that it's rare".) If you want a Bayesian response, what is your prior? Or did you just want to know the probability of finding data as extreme as yours under the assumption it's rare? Etc. $\endgroup$
    – gung - Reinstate Monica
    Commented Jan 21, 2016 at 0:08
  • 1
    $\begingroup$ This seems like a question for @cardinal... $\endgroup$
    – gung - Reinstate Monica
    Commented Jan 21, 2016 at 0:08
  • 1
    $\begingroup$ I am a high-school dropout, so Poisson - yeah, homogenous - yeah. No natural starting point - observation can start at any time. Since there's no answer from frequentist point of view, let's see Bayesian. Don't have any priors, and hope to stay that way :) $\endgroup$
    – Grk58
    Commented Jan 21, 2016 at 4:04
  • $\begingroup$ I want to see how to approach this very generic problem from the statistician point of view (which I'm obviously not). In other words, I'd like to order a sandwich without listing every ingredient, relying on your, chefs, expertise. $\endgroup$
    – Grk58
    Commented Jan 21, 2016 at 4:06
  • $\begingroup$ Can't get a Bayesian answer without priors :) $\endgroup$
    – Mageek
    Commented Jan 21, 2016 at 5:26

1 Answer 1

Reset to default
1
$\begingroup$

At a high level, no, seeing an event on day 3 followed by one on day 80 should not cause you to reject the idea that the event occurs on average 90+ days apart. This is an important point. I am assuming that you mean 90+ on average. Obviously if it were assumed always 90+, then you would reject that hypothesis.

Discrete events are often modeled with an exponential distribution (i.e. a Poisson process as mentioned by Gung). Assuming a process with a mean of 90: $$f_x(t)=\frac{1}{90}*e^{-\frac{t}{90}}$$

The probability of two events being interspersed by only 77 days (given assumptions above is):

$$P(0\leq t \leq 77)=\int_{t=0}^{77}(\frac{1}{90}*e^{-\frac{t}{90}})$$

$$P(0\leq t \leq 77)=1-e^{-\frac{77}{90}})$$

$$P(0\leq t \leq 77) \approx 0.575$$

We generally reject hypotheses when the probabilities are under 0.05.

Had you seen a much smaller interval, or better yet a long list of smaller intervals, perhaps we could reject the 90+ day hypothesis, but based on these data points, we certainly cannot. Note also that the mean of 90 is affected by the long tail of the exponential curve, and thus the median is lower $90\ln{2}$ or about 62 days between events.

P.S. I loved your comment about priors, but it might have gone over a head or two in this community ;)

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks, @MikeP! Makes sense and actually useful. Does the first point get the same treatment, e.g. if I only have 3 days of observations and see the event, the probability that it's rare is 0.03? It doesn't seem right. $\endgroup$
    – Grk58
    Commented Jan 21, 2016 at 19:30
  • $\begingroup$ No problem @Grk58! Since there is no natural starting point, we don't know how long we've waited (or would have were we to have been observing) before day 3. Without the natural starting point, n events only gives us n-1 "data points." $\endgroup$
    – MikeP
    Commented Jan 21, 2016 at 20:06
  • $\begingroup$ which makes a probability what? $\endgroup$
    – Grk58
    Commented Jan 21, 2016 at 20:23
  • $\begingroup$ you cannot calculate a probability for the interval based on a single event unless its referenced to some natural starting point. $\endgroup$
    – MikeP
    Commented Jan 21, 2016 at 20:24
  • $\begingroup$ gotta love stats! the only discipline where "cannot do" is a perfectly acceptable answer :) still need a solution though. Intuitively, this probability is very small when the observation starts, and approaches 1.0 as the observation period goes over 90 days. $\endgroup$
    – Grk58
    Commented Jan 21, 2016 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.