After how many events can you say the failure rate of one piece of equipment is greater than another?

Question

If there exist two identical pieces of equipment, in this case two pumps that are the exact same (in theory) pumping the same fluid in the same location at the same rate (everything is the same), how many times does pump A have to fail/breakdown while pump B is fine before I can say that there is a problem with pump A and NOT pump B, rather than saying there is clearly a problem and it exists with both pumps (or saying I can't pass judgment either way).

That is, if pump A failed 50 times in the past 10 years and pump B failed 0 times, that probably means whatever caused pump A is likely unique to pump A. However if pump A failed only 1 time in 10 years, and pump B still never failed.. I'm assuming we'd call that statistically insignificant and I couldn't say either way whether or not the problem is with pump A or pump A AND B.

My thoughts: At first I thought I'd need to know what the nominal failure rate of that type of pump was before being able to figure this out, but then I thought, "if these pumps normally fail 5 times a year, then there's got to be a difference because A is following that trend exactly, but B is 50 failures away from where it should be" and conversely the same logic can be applied at the other extreme, say if the pumps normally fail once every 20 years.. there's still quite clearly a discrepancy.

Answer 1

As I understand your situation, you have just two numbers: after a given amount of time, you have the number of times A has failed, $Y_A$, and the number of times B has failed, $Y_B$. These are counts.

It is possible to test two counts, but only if you make strong assumptions. You need to assume an underlying distribution for the counts. There are a number of count distributions, but the most basic is the Poisson distribution. It is not the most realistic possibility. However, a feature of the Poisson distribution is that the variance / SD is a function of the mean. Moreover, a single count constitutes an estimate (not necessarily a great one) of the underlying mean. In other words, if you are willing to assume that the counts are distributed as a Poisson, you can conduct a test (called a Poisson test) even though you have only two numbers. Obviously you would have better power if you had more than two numbers. In addition, the validity of the outcome is dependent on the veracity of the distributional assumption. It is typically more realistic to think of the output from a Poisson test as the lower bound of the significance, with the actual significance being some unknown amount higher.

For example, we can run a Poisson test in R using your data:

poisson.test(c(50, 0))
#  Comparison of Poisson rates
# 
# data:  c(50, 0) time base: 1
# count1 = 50, expected count1 = 25, p-value = 1.776e-15
# alternative hypothesis: true rate ratio is not equal to 1
# 95 percent confidence interval:
#  13.0604     Inf
# sample estimates:
# rate ratio 
#        Inf 

Answer 2

I'm not sure about the math, but something that pops out to me is that it must be taken into consideration that the probability of failure is not static - probability of failure over time must increase, on the system and therefore the individual components. If you want to just call something going wrong "failure", that's fine, but you do have to recognize that failure is the result of one or more parts failing. If you want to say that the design is out of spec, that's different, and has to be measured not by failure but by taking the pump apart and measuring things (I think).

If you are holding the probability of failure fixed over time, then you either know p(x) or are estimating p(x). In either case that is a simple comparison, but I think you will always need p(x). Any thoughts? I could be way off.