# How could one get a difference between expected and observed probabilities with rare events?

Say we have a car with an electronic ignition system. Our engineers have deemed that due to mechanical failure possibilities, there is a 1 in a billion (or some huge number) chance that the car will explode on ignition. Unbeknownst to our engineers, a circuit design flaw means that the real probability of explosion is 1 in a thousand. In accordance with this, in the total thousand tests, the car blows up exactly once. Since this just a one off event, by what means could our engineers figure out that their probability estimate was off from just one event? Assume that they do not want to do more tests, as the explosions are expensive and dangerous. From the 1 in a thousand number, they want to change the probability of explosion to a higher value.

Making things more absurd, what if the explosion happened on the first attempt? This is rather unlikely, but could anything be done with a sample size of 1? If not, how does repeating the experiment and getting more "no explosion" results make the higher rate of explosion more credible?

• You are describing 2008 financial crisis here Commented Jan 30, 2022 at 14:51

I would approach this differently. Instead of testing your prior beliefs I'd update them.

Here's example. Suppose before the experiment you spoke with experts and they conveyed their beliefs about the parameter as $$\lambda\sim\Gamma(\alpha=10^{-7},\beta=0.1)$$, i.e. from Gamma distribution with mean $$\mu=\alpha/\beta=10^{-6}$$ and variance $$\sigma^2=\alpha/\beta^2=10^{-5}$$.

I chose Gamma because it's a conjugate prior to Poisson, so it's easier to do paper math for this example. In practice you can do numerical math or simulations to do the same with arbitrary priors. Also, Gamma and Beta are convenient parametric distributions to capture expert opinions on unbounded and bounded domains.

You conducted an experiment and observed only one event in $$n=1000$$ trials. This corresponds to MLE of Poisson parameter $$\hat\lambda=\bar y=\frac 1 {1000}$$.

Let's update the prior with our sole event detection, and get the posterior distribution: $$\hat\lambda|Y=\Gamma(\alpha+n\bar y,\beta+n)=\Gamma(\alpha=1.0000001,\beta=1000.1)$$

Now, our new mean is $$\hat\mu\approx 0.9999\times 10^{-3}$$ and its variance is $$\hat\sigma^2\approx 10^{-6}$$ - a MASSIVE change.

This is just an example of the approach, and it's not the only way to do this kind of things. You could use exponential distribution, for instance, to model failures. You could perform survival analysis etc.