1
$\begingroup$

I'm studying widgets and their failures. Generally a widget will run for many years without trouble, but 1-2% of widgets will fail in a given year. I have a table which lists widget manufacturers (A, B, C, ...), number of widgets from that manufacturer, how long they have been observed to run, and how many have failed. I want to estimate reliability of each manufacturer's widgets.

The straightforward thing to do is divide the number of failures by the number of widget-years observed to get a crude failure rate $\hat{\mu}$. But I'd like something that takes into account my general knowledge of the overall failure rate across all manufacturers (1-2%) as a sort of prior, which would help when there are relatively few widget-years of observation. This shows up especially when no failures have been observed for a given manufacturer and $\hat{\mu} = 0$, a kind of violation of Cromwell's rule (because if you use this as a prior in any future calculations, you'll remain trapped at 0, regardless of how strongly the information signals in the other direction).

So what is the best way to find a failure rate that combines the information that I have from these failing widgets (Bernoulli trials)?

$\endgroup$
3
$\begingroup$

It depends on exactly how precise you want to be. Intuitively, we should expect the strength of the observed evidence to be proportional to the square root of sample size, so $\hat{\mu}=[\bar{x_s}\sqrt{n}+{\bar{x_p}}]/[\sqrt n +1] $ where $n$ is the sample size, $\bar{x_p}$ is the overall observed mean, $\bar{x_p}$ is the sample mean for the subset, and $\hat{\mu}$ is your estimate of the subset mean, will be good enough for a rough rule of thumb, and will be sufficient to both satisfy Cromwell's Rule (for $n=0$, you just use $\bar{x_p}$, and as long as $\bar{x_p}>0$ and $\bar{x_s}\ge 0$ , $\hat{\mu}$ will be positive), and for large $n$ your $\hat{\mu}$ will go to $\bar{x_s}$.

One can argue that there should be a more complicated expression than just $\sqrt n$, but if you want a precise calculation, you need to have not only a prior distribution on $\mu$, but also a prior for how likely you think it is for a subset to be different. For instance, if you find that the sample mean for widgets that are exposed to high temperatures is different from the general population, you might want to give a stronger weight to that than if the sample mean for widgets made on Tuesday is different.

Note that I said "distribution". Bayesian updates take evidence $E$ and hypothesis $H$ and compare $P(E|H)$ to $P(E|\neg H)$. You seem to be thinking of the number of failures as $E$ and whether the next widget will fail as $H$. But that doesn't work: if each widget fails independently, then $P(E|H) = P(E|\neg H)$. Instead, what you have to do is, for each possible value of $\mu$, assign a value of how likely you think it is that that is the correct value. It is then this distribution, rather than a particular estimate of $\mu$, that gets updated. So you'll start with a prior distribution for widgets in general, update that based on the overall failure rate, use that distribution to create a prior distribution for a particular manufacturer, then update that distribution based on the data for that manufacturer. There is the further complication that if the overall failure rate includes data from the manufacturer currently under consideration, then you're "double counting" that data, and there will be independence considerations. You can find a discussion here on how to use Beta distributions to model uncertainty regarding failure rates. Or do a web search on "bayesian updating binomial distribution".

$\endgroup$
  • $\begingroup$ Very interesting, thank you. This is a lot to mull over. It strikes me that if I have a Poisson process, the rule treats widget-years differently depending on whether they come from the same widget or different widgets. That's not obviously right, and yet I can't find a good way around it (since there's no reason to favor widget-years over widget-seconds or widget-millennia). Thoughts? $\endgroup$ – Charles Jul 10 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.