4
$\begingroup$

I've never taken a stats course so I really don't know where to begin on this.

I am using microcontroller that according to the datasheet, has a Flash memory that can be reprogrammed a minimum of 10,000 times, with 100,000 times being a more typical number.

These numbers seem low to me (based on other devices using a similar technology which claim lifetimes of an order of magnitude higher), so I decided to take one of our products and do destructive testing on it. I started testing last Friday night and it is already up to 600,000 events without an error (as determined by calculating a 16-bit CRC over the 4K bytes of data test being Flashed). Based on this device far surpassing the "typical" value in the datasheet, I have started tests on three more devices.

So at some point I will have a failure value for all four devices. How can I use that data to predict an expected lifetime x for the device with a 99.9% certainty? (Not sure if I am saying that right, what I mean is that only 0.1% of devices will fail before x events).

Improve this question
$\endgroup$
5
  • $\begingroup$ The traditional way you could do this is through hypothesis testing. This will allow you to test whether the plausibility that the lifetime is equal to some value, say 10,000 times. en.wikipedia.org/wiki/Statistical_hypothesis_testing $\endgroup$
    – Christopher Aden
    Commented Jan 12, 2011 at 1:39
  • 3
    $\begingroup$ You have to make strong assumptions about the lifetimes. Without them, you would need to test approximately 3/(0.1%) = 3000 devices. For example, suppose 1% of the devices have a relatively rare defect that causes them to fail after 100 reprogrammings. Your chance of encountering one of them during your testing of only four devices is approximately 4%: pretty low. Thus, based on the data alone, you can have very little confidence that such defects don't exist in the population of microcontrollers. $\endgroup$
    – whuber ♦
    Commented Jan 12, 2011 at 2:35
  • $\begingroup$ @whuber - is my answer consistent with your point about assumptions? $\endgroup$
    – David LeBauer
    Commented Jan 12, 2011 at 5:20
  • 1
    $\begingroup$ @David Yes it is. Good work. I was thinking along the lines of borrowing strength from other datasets or knowledge about failure modes and distributions derived from similar devices. Such information could justify a Bayesian estimate, too. $\endgroup$
    – whuber ♦
    Commented Jan 12, 2011 at 15:19
  • 1
    $\begingroup$ It may be there is a significant variation in chips from different wafers (batches), but low variation within a batch. This means that if the four chips are all from the same batch, it may give a misleading view of the lifetime of a chip selected at random from the whole production run. It may be the low lifetime quoted is because there is occasionally a bad batch and it is easier to quote a low life expectancy than to detect the bad batches. If your chips are all from the same batch, it may not be possible to estimate the true population lifetime. $\endgroup$
    – Dikran Marsupial
    Commented Jan 12, 2011 at 18:28

1 Answer 1

Reset to default
5
$\begingroup$

@whuber makes an important point about assumptions,

It is common to assume that failure occurs either because a part is defective or because it has worn out. This means that there are two processes causing failure, and defective units fail shortly after deployment whereas wearing out happens over time. It is difficult (or impossible) to justify parameterizing both of these processes if $n=4$.

Case 1

For a simple case, assume that defective units can be eliminated (e.g. by an initial screening or 'burn in'), and all units fail due to a constant wearing process. This assumption may be valid since the data sheet says that the minimum expected lifetime is $10,000$ events. One might model this process as an exponential distribution:

Example: assume that your four test units fail after $[5,10,20,40]x10^{5}$ reprogrammings. You can find the maximum likelihood estimate for the exponential distribution in R:

library(MASS)
fitdistr(c(5, 10, 20, 40)*1e+5, 'exponential')
lambda <- fitdistr(c(5, 10, 20, 40)*1e+5, 'exponential')$estimate

To find the time during which only 1/1000 units will fail, estimate the 0.1th quantile of this distribution.

qexp(0.001, lambda)

In this example, the result is an expected lifetime of 1875.

edit As whuber points out, $n=4$ is still to small for this problem, because it is very difficult to estimate small probabilities, and distribution tails are very sensitive to assumptions about the probability distributions used to model the data, and to the data itself.

Since you have good prior knowledge from the product datasheet, you could incorporate this information into your analysis as a prior on lambda. The gamma distribution is a conjugate prior, $\lambda\sim \text{Gamma}(\alpha, \beta)$, but I can not find a parameterization appropriate to the prior information, with $\text{median}\simeq 100,000$, $P(Y<10,000)\simeq 0)$ and nonzero density above $10,000$. Once you determine the model to use, it would be worth asking a separate question.

Case 2

If you assume that failure is caused either by defect or by wear, you could consider the Weibull and Generalized Exponential distributions (Gupta and Kunda 1999, 2001, 2007). The field of survival analysis provides many options, but as whuber points out, $n=4$ is insufficient to justify a model other than, perhaps, the single parameter exponential.

Gupta, R. D. and Kundu, D. (1999). Generalized exponential distributions", Australian and New Zealand Journal of Statistics, vol. 41, 173 - 188.

Gupta, R. D. and Kundu, D. (2001), Generalized exponential distributions: different methods of estimation", Journal of Statistical Computation and Simulation. vol. 69, 315-338.

Gupta, R. D. and Kundu, D. 2007. Generalized Exponential Distribution: existing results and some recent developments. Journal of Statistical Planning and Inference

Improve this answer
$\endgroup$
11
  • 1
    $\begingroup$ +1 for the R code illustrating fast solution of the problem $\endgroup$
    – mpiktas
    Commented Jan 12, 2011 at 7:40
  • $\begingroup$ @David if $n=4$ is too small to justify most models, why is it enough to justify any model? $\endgroup$
    – whuber ♦
    Commented Jan 12, 2011 at 15:21
  • 1
    $\begingroup$ @David I don't know enough about this particular situation to comment on any specific proposal. The point I am making is that there is no magic in statistics that will generate "99.9% certainty" from a sample of four. A single-parameter model might fit beautifully but anybody would be foolish to rely on a 99.9% CI or percentile estimated with it unless they had substantial, independent evidence that the model is appropriate. $\endgroup$
    – whuber ♦
    Commented Jan 12, 2011 at 16:04
  • 1
    $\begingroup$ @David I didn't plead ignorance concerning the subject matter :-) I just don't know what additional information the OP has, so I have no idea whether your gamma prior is reasonable. (Choice of prior is a big deal with this small sample.) The concern is only indirectly about high uncertainties attached to tail probabilities. More fundamentally it's about the model itself: if you use a bad model for the failure distribution, you will likely get bad (or awful) estimates. How, then, can you tell whether the model is appropriate? Not by testing just four units! $\endgroup$
    – whuber ♦
    Commented Jan 12, 2011 at 16:36
  • 1
    $\begingroup$ @tcrosley yes, that is correct; MASS is one of the best books for learning R - I don't think that it would be over your head. I used MASS for the fitdistr, which provides a maximum likelihood estimate (MLE) of a distribution's parameters given data. So, if you are interested in the theory, use ?fitdistr and read up on MLE. But you might want to consider comparing the fits of different distributions, perhaps contacting the manufacturer for additional data that could be used in a hierarchical model. $\endgroup$
    – David LeBauer
    Commented Jan 12, 2011 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.