2
$\begingroup$

I have a complicated physical model that can produce a certain quantity (real-valued) for a large (in the 1000's) number of points (N). We assume that mean and variance of the model output is a good representation of the actual physical object properties but we don't know as we have not built them yet (please let me know if this a bad assumption). So $\mu_{model}=\mu_{population}$ and $\sigma_{model}=\sigma_{population}$. From here, I'm trying to find the probability that the mean of a relatively small sample ($n$ < 10) is below a certain limit $L$.

First basic approach is to use the square-root law. Basically I can convert my z-score from the model/population into a z-score for my sample: if I had $\frac{\mu-L}{\sigma}=-1.15$, assuming normal distribution, while a single object would have had 75% chance of having its property below the limit $L$, the mean of a sample of $n$ objects would have a z-score of $-1.15\sqrt{7}=3.04$ and thus a probability of 99.8% to be below the limit $L$. I feel like this is the right trend as for a very large sample, the sample mean should converge to the population mean and thus the sample mean should have near certainty to be below the limit.

Another approach is to use a t-score. $CDF_t\left(\frac{L-\mu}{\sigma}\right)$, here again using the population mean and std rather than the sample ones. So basically $CDF_t(1.15,6)=85.3\%$ certainty to be below the limit for the mean of my sample of 7, but what does not feel right is that $CDF_t(1.15,n)$ does not tend to 100% as n goes very large.

Another brute force approach would be to build samples of my modeled population and compare the mean of these samples to my limit $L$. Not sophisticated but maybe I should leverage the fact that I have an actual (large-ish) 1000+ actual members of the population and not make any other assumption.

Is any of these methods correct? If not what would you suggest I use?

Edit: I have a machine (say a gas turbine) that takes air and fuel at certain (T,p) conditions (bounded) and transform it into a different fluid at different (T,p). This gas turbine has 10's of internal parameters that defines it, we'll call these (x1,x2,...,xn). I'm only interested in one output Y and I have built a model to predict Y = f(air_T, air_p, fuel_T, fuel_p, x1, x2, ..., xn). For a given built machine (which means x1, x2, ..., xn are fixed), we are interested in the statistics of Y across a range of inlet conditions. We assume that there is enough physics in that model that it gets the mean of the quantity Y correct, but probably not the higher order statistics. I want to find the probability that the quantity Y across a small sample of machines (but each across a large number of inlet conditions) has a mean below a certain limit L.

$\endgroup$
5
  • 1
    $\begingroup$ At present your introductory explanation in Para 1 is unclear and requires elaboration. (Question cannot be closed since it is bountied, but unlikely you will get answers in present state.) Rather than focussing on solution methods, please add clear explanation of the initial problem. $\endgroup$
    – Ben
    Oct 16, 2023 at 22:35
  • $\begingroup$ Is the small sample $n<10$ from the actual physical object? $\endgroup$
    – Idontgetit
    Oct 17, 2023 at 8:03
  • $\begingroup$ In part of your question you assume a normal distribution - is this assumption justified? Despite the complicated nature of the physical model? $\endgroup$
    – Idontgetit
    Oct 17, 2023 at 9:06
  • $\begingroup$ "We assume that mean and variance of the model output is a good representation of the actual physical object properties but we don't know as we have not built them yet (please let me know if this a bad assumption)." This cannot be addressed without knowing the physical background and in what sense you want it to be "good". $\endgroup$ Oct 17, 2023 at 21:11
  • $\begingroup$ Sorry I was away for a few days... As far as a clearer explanation of the problem, let me try again as an edit to the initial post. The small sample means that if I actually built less than 10 of these objects, what would be the mean and variance of the performance metric I'm interested in for that object. Normal distribution is pretty good for some of the inputs of the random variables of the model, but probably not for the output. $\endgroup$ Oct 24, 2023 at 5:47

1 Answer 1

5
+100
$\begingroup$

Assume that you have a process which generates normally distributed random variates with mean $\mu$ and variance $\sigma^2$.

Goal is to find the probability that the mean of a sample $\bar{x}$ is less than a limit $L$. Let $s$ be the sample standard deviation.

$$P(\bar{x} < L) = P\left(\frac{\bar{x} - \mu}{\frac{\sigma}{\sqrt{n}}} < \frac{L - \mu}{\frac{\sigma}{\sqrt{n}}} \right)$$

$$\frac{\bar{x} - \mu}{\frac{\sigma}{\sqrt{n}}} \sim N(0, 1)$$

Therefore

$$P(\bar{x} < L) = StandardNormalCDF\left(\frac{L - \mu}{\frac{\sigma}{\sqrt{n}}} \right)$$

if $L > \mu$, as $n \rightarrow oo$, $P(\bar{x} < L) \rightarrow 1$

Once you take your sample, and want to compute the confidence interval of the mean, you will use the t distribution

$$P(\bar{x}-t_{n-1,1-\alpha/2}\frac{s}{\sqrt{n}} < \mu < \bar{x} + t_{n-1,1-\alpha/2}\frac{s}{\sqrt{n}}) = 1-\alpha$$

Based on the question, this is all that is necessary when you assume that the simulation output is the true mean and standard deviation of the the "population". The answer would have more variance if the model was stochastic and you also took into account the error with the model mean and variance.

$\endgroup$
1
  • 1
    $\begingroup$ a small note: t-dist for independent normal measurements will be relevant when $\sigma$ is estimated using $s$ in your notation; this is just to highlight why t-distibution is used in the CI calculations. If $\sigma$ is known, Gaussian law should be used, of course. +1 $\endgroup$
    – Math-fun
    Oct 24, 2023 at 6:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.