Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
Let's say we run a simulation where thousands of measurements happen, each measurement has a numeric value. For each simulation we output a single aggregate value of all the measurements in the form of a p90.
If we run N simulations and gather N output values, can we determine a confidence interval for this data?
Does the fact that the individual output values are actually an aggregate percentile value change anything here? Is there a special name for this?
There isn't anything mathematically "wrong" about this, but you just would need to be careful about your interpretation. Essentially you are using your p90 score as a latent variable to represent something useful. This is really no different than, say, doing a confidence interval on SAT scores, which are themselves an aggregate score based on lots of other scores (i.e. the questions on the SAT).
The real trouble people get into in this area is over-inferring, and trying to say something about their measurments. Your confidence interval is only on your aggregate score, and you can't try to make inferences about your original measurments.
