Understanding the probability distributions behind a Monte Carlo experiment

Question

A colleague and I are trying to model the expected maintenance cost/h (E[C/h]) of a component A on an aircraft over its life based on its reliability distribution. As the component fail, it's replaced by the same component but new and each component life span are independent from each others and follow the same distribution.

A follow a Weibull distribution with 2 parameters: (beta = 2.1 and alpha = 2500). We know (and have observed) that both models converge to the same constant value that we call "Mature Cost / hour" (MCH).

MCH = Cost / MTBF, but we are interested in the transition phase leading to that plateau.

We took 2 different approaches, that yield 2 different results and we are trying to understand the differences.

----- APPROACH #1: -----

E[C/h(t)] = P(A fail <= t) x MCH. where P(A fail <=t) is the cumulative Weibull distribution

The idea being, the 1st component comes for "free" with the aircraft (no maintenance). Once it has been removed at least once, and the full cost of the component has occurred, you can assume subsequent removals to occur on average every MTBF.

-----APPROACH #2:------

Use a Monte Carlo simulation with a fleet of 10,000 components, all starting at the same time, and we count how many removals occurs in a specific interval of time T = (t; t+dt).

E[C/h(T)] = Sum of removals in the interval(T) x Cost / (number of hours in the interval x 10,000).

-----RESULTS-----

The green curve is Approach 1

The blue curve is Approach 2 (each interval is 100 hours) and this represent the expected cost in this interval only.

The orange curve is Approach 2 but this time it's the cumulative number of removals and hours since the beginning that we use.

-----QUESTION-----

Approach 2 (Monte Carlo) constantly over estimate the cost in the beginning compared to the Approach 1.

My theory is, by simulating thousands of component with the same distribution a lot of the first removals will tend to occurs around the same value. Hence there is a concentration effect that exist with many component but does not exist when looking at a component in isolation.

My colleague theory is that the MC is overshooting because for a specific interval T, it's looking at the possibility of the component failing once but also the probability that, in extreme case, the component could have failed 2 or 3 times at that interval. Hence the MC is more "accurate" because he accounts for this scenarios as well.

Is my colleague right in assuming that he can use the results from 10,000 component to asses the behavior of 1 and that the MC is providing additional insight on the component ? Or are both approach modeling different thing altogether ?

Answer 1

Stuck with MC results if you validated your MC algorithms/codes

I have worked for aviation industry for many years as an system safety & reliability engineer.

• 1 item per aircraft or does it have redundancy?
• Does any pre/post flight check for item A exist?
• What is the schedule of the fleet? What are the mission (flight) hours for each?
• Remove/Replace costs will differ in different airports due to parity ratio and time-to-replace differences probably. What are these ratios?
• if this item is safety critical & has redundancy, then you have a chance also failure of all A items & the catastrophic result. This is a cost also.
• and may be more constraints specific to item A...

One of the most beautiful results with MC is you'll have %n confident probability values like:

• required spares per airport in order not to be in spare-shortage with a probability of %95 is n
• or, with a probability of %95, expected total cost for the maintenance of item A will be in $[C1,C2]$ for the new fleet flight schedule of 2023 is X.
• ... much more possibilities of reporting

For your models:

• "The blue curve is Approach 2 (each interval is 100 hours)" seems problematic to me since 100 >> An average flight (mission) hours. It seems like a random choice to see smaller numbers which probably hurts the reality level of the model.

• "MC is looking at the possibility of the component failing once but also the probability that, in extreme case, the component could have failed 2 or 3 times at that interval". If the item A has redundancy, it must consider the failure of any or more of them. This is what should be done actually.

• If the item A has no redundancy (single A per aircraft), then your MC code mustn't allow to produce $>1$ failures for item A. (validated MC code!)

• 10,000 component to asses the behavior of 1 and that the MC is providing additional insight on the component. If the meaning of the "of 1" phrase is "items A in the fleet" then yes, MC is providing additional insights on the component in the fleet domain. Else if the meaning of the "of 1" phrase is the item A solely, then no. (No additional info for item A that it's remove/replaceable with predicted Weibull parameters.)

• APPROACH #2 is ok since it doesn't use faulty/useless MTBF concept. observe that even I said that 100 hours choice in blue model may be faulty, it's more compatible with orange model.

• Stay away from green model. No point to know Cost per MTBF since time between failures varies with a Weibull distribution with an aging unit. ($\beta>1$). Generally, stay away from MTBF. MTBF is a too strong blundering illness in reliability engineering world without any vaccine for masses.

• Lastly, you may wish to compare your MC results (I recommend) for various number of runs. For example results for 1e+3 runs, 1e+4 runs, 5e+4 runs etc.. to feel ok that the MC converged enough and increasing the number of runs won't add any plus value. Good luck with your decisions.