Reliability engineering: Measuring the proportion of defective products at several time points using the same test units A product being designed will have a failure-free warranty requirement of 1 year and the Engineers are planning a series of tests to estimate our warranty exposure.
Based on the binomial confidence interval analysis procedure described in section 7.2.4.1 of the NIST/SEMATECH e-Handbook of Statistical Methods I have computed the following in advance of the test. I used the exact intervals for small numbers of failures.


*

*If out of a random sample of 20 units, 4 units fail the 1 year warranty simulation test, I believe we can say that we are 90% confident that the population defect rate will lie between 7.1% and 40.1% 

*If out of a random sample of 20 units, 1 unit fails the 1 year warranty simulation test, I believe we can say that we are 90% confident that the population defect rate will lie between 0.26% and 21.6% 

*If out of a random sample of 20 units, 0 units fail the 1 year warranty simulation test, I believe we can say that we are 90% confident that the population defect rate will be 13.9% or less.

*If out of a random sample of 1 unit, zero units fail the 1 year warranty simulation test, I believe we can say that we are 90% confident that the population defect rate will be 95% or less.


Now assuming no failure occurs at the 1 year point in the test(s), continuing the test(s) beyond the 1 year warranty period with a some corresponding benefit in confidence would be desirable as the test units are costly to build. Can the above analysis procedure be extended to answer the following questions when the simulation test time exceeds the actual warranty period? 


*

*At the 90% confidence level, what are the upper and lower limits of the population defect rate if a 1.5 year warranty simulation test is run on 1 unit and that unit does not fail?

*At the 90% confidence level, what are the limits of the population defect rate if 2 units are warranty simulated. One unit fails at 1.5 years and the other fails at 4 years?

*Can Monte Carlo be used here to help answer/validate these questions?
Many thanks for any guidance.
 A: With respect to the underlying statistical technique, whether the duration is shorter or longer than the warranty seems immaterial. You are just making inference on a proportion and you can indeed use the same technique for another point in time (your question 1).
That said, the time points need to be specified in advance. You should not collect data continuously and then choose the point at which you compute a confidence interval based on these data. For example, you should not:


*

*Simulate two years of product life, notice that two units fail after 20 months and compute a confidence interval at 19 months purporting to show that reliability is high at this point in time.

*Compute many tests/confidence intervals (say one every month or more) and just pick one that suits you (say the last one with an upper bound under a particular threshold or something like that).


In both cases, you are increasing the error level by (implicitly or explicitly) running many tests and the confidence intervals will be misleading.
A better way to approach this if you want something else than a confidence interval for a pre-specified time point (and an answer to your question 2) is to model the time-to-failure instead of the proportion of defective/failure rate. Survival analysis techniques are also relevant.
