Estimate the number of failing components in a changing population

Question

I'm working on a problem (using R) which involves estimating the number of failures in a population of components. I have information on the number of components that were added to the population in each month & the number of failures in each month (over a period of 28 months). Of course I also know age of any component at any given time (in days). This diagram gives an idea of the current age-profile of the population. The failures are included in red (they occur rarely).

Here is a histogram with the age-profile of the failed components

It appears that failure is approx. equally likely at any time (the diagram above suggests that most failures occur in the early stages, < 250 days, but the population contains many more "young" components than old)

Here is info on how the population developed over time:

date,new_components,cumul_new_components,failures,cumul_failures
2013-06,1,1,0,0
2013-07,3,4,0,0
2013-11,64,68,0,0
2013-12,154,222,0,0
2014-01,69,291,0,0
2014-02,5,296,0,0
2014-03,110,406,0,0
2014-04,92,498,0,0
2014-05,424,922,1,1
2014-06,355,1277,0,1
2014-07,402,1679,1,2
2014-08,589,2268,0,2
2014-09,763,3031,1,3
2014-10,601,3632,3,6
2014-11,550,4182,3,9
2014-12,762,4944,2,11
2015-01,632,5576,8,19
2015-02,638,6214,1,20
2015-03,883,7097,1,21
2015-04,845,7942,2,23
2015-05,1263,9205,4,27
2015-06,1374,10579,8,35
2015-07,1357,11936,7,42
2015-08,1686,13622,8,50
2015-09,1384,15006,11,61
2015-10,1920,16926,7,68
2015-11,1739,18665,19,87
2015-12,1978,20643,18,105
2016-01,216,20859,0,105

I want to predict how many failures will occur in the next 3 months, with a confidence interval. I tried to fit a Weibull distribution but I'm not sure it's so suitable given how few failures occur. Are there maybe better (non-parametric perhaps?) methods available that might allow me to estimate the number of failures in the next 3 months? I expect the population to grow in the next 3 months also, if I could allow for this in the estimation that would be even better. Thanks in advance :)

EDIT Adding the full dataset here
EDIT #2 I returned to the Weibull distribution after looking at some survival function graphs & realising that the components showed a slight wearing out behaviour. I also discovered that the Weibull was suitable for handling cases with relatively few failures. Below is the Kaplan-Meier survival curve. It's clear that the curve begins to dip a bit around the 600 day mark.

I then fit a Weibull distribution to the data using the survival package in R.

srFit_weibull <- survreg(Surv(age, delta) ~ 1, dist="weibull")

Where delta is 1 for events, i.e failures & 0 otherwise (I've added delta to my original dataset). Inspecting srFit_weibull I get the parameters.

Coefficients:
(Intercept) 
9.822805 
Scale= 0.8101698 

Now, from this answer I need to convert these parameters in order to get the cumulative distribution function (cdf), i.e. F(t)

$\beta = 1/0.8101698 = 1.23$
$\eta = exp(9.822805) = 18449.73$

And the cdf F(t) = $1 - \exp(-(\frac{t}{\eta})^\beta)$

Now my question is
(i) with F(t) how do I get an estimate, with confidence interval, of how many failures will occur in the next 3 months within this current population? I've read that the number of failures is simply $n*F(t)$ but I'm not sure if this holds in my case (as the population contains components of varying ages & the components exhibit a wearing out behaviour)

(ii) If I add 2000, 2200 & 2500 new components to the population over the next 3 months how can I get an estimate, with confidence interval, of how many components will break over the coming 3 months?

Answer 1

I'll read quite a bit into your problem, so please correct me if I over-read.

You have components produced in a manufacturing process, and these are indistinguishable (lacking a lot number or any other such feature), so failures inform you only about 'the component' in general, and not specific groups of components. When failures occur, some cost is incurred that you need to prepare for; maybe, for example, some kind of urgent service is required to replace the failed part, and you need to allocate a sufficient pool of 24/7 service personnel to meet the anticipated workload.

Your analysis is divided into 2 stages. In the first stage, you characterize the survival function $S(t) \equiv 1-F(t)$ of the component. You could consider this stage the 'engineering' stage. (Consider that this is the sort of thing you might even be able to read off a suitably detailed 'spec sheet' for the component.) In the second stage, you want do the 'operations research' (or, simply, 'business') side of the analysis, and you want to estimate the service-cost of a planned schedule of deployment.

It sounds to me as if the 'engineering' stage of the analysis is already done, and you are reasonably happy with your characterization of the failure behavior of this component, using a Weibull distribution. The 'business' side of the analysis might profitably be prototyped in a spreadsheet before you implement it in R.

Consider first the expected number of failures at some time $T$, 3 months in the future. The relevant calculation is:

$$ \Sigma_{i=1}^{N} F(T-t_i), $$

where $i$ ranges over all the components $\{1,..,N\}$ you are planning to deploy before some date $T$ (say, 3 months into the future as per your expressed concern), and the $t_i$ are the times at which each component were/will be deployed (where $t_i<0$ for an already-deployed component). This is just the sum (over all the components that will be fielded as of $T$) of the probability that the component will fail by time $T$. The spreadsheet would have one row for every component, and 2 columns: one column for the (past or planned future) deployment date $t_i$; a second column with $p_i(T) \equiv F(T-t_i)$, the probability that component will fail by time $T$.

Your desire for a confidence interval around this expectation essentially means you want to know the distribution of the number of failures by time $T$. To me, the obvious way to do this would be by a simulation: Add another M=200 columns to the spreadsheet -- titled 'rep1', 'rep2', ..., 'rep200'. Fill each of these with 0/1 values according to random Bernoulli draws with probabilities $p_i$ in column 2 of the spreadsheet. At the bottom of this big $N \times M$ grid, sum the columns to get a row of failure counts $n_{rep}$. Plot a histogram of that row. Sort the row, and pick off the 10th and 190th elements, and say, "If our Weibull time-to-failure model is correct, then we have a 95% chance of seeing between $n_{10th}$ and $n_{190th}$ failures in the next 3 months."

I should note that, because your failures are so rare, I felt entitled to ignore the possibility that a component replaced by the service department will subsequently fail. The analysis I've suggested cannot be extended to a period $T$ comparable to the mean component life.

Finally, you should keep in your back pocket the simple, exponential decay approximation to your problem, in which each component acts like a radioactive atom with a constant failure rate. Making time thus homogeneous introduces much simplicity and enables closed-form solutions that you can (and should) use to check any result you get from your Weibull model.