Within Statistics, I have seen the following applications of simulation:
- Bayesian Sampling: When the Posterior Distribution can not be analytically integrated, we use MCMC algorithms (e.g. Metropolis-Hastings) to "simulate" points from the Posterior Distribution
- Model Misspecification: Suppose I create a statistical model where I assume that the response variable has a certain distribution (e.g. Exponential Distribution) - I can simulate data from different distributions (e.g. Gamma Distribution) and see how "robust" my model is to conditions outside of what the model was created under
However, I keep hearing that it can be important to simulate data from the same model itself - for example, I fit a specific regression model on a specific dataset, I might need to simulate data from this regression model itself. I am trying to understand why this might be important and useful.
As an example, suppose there is a school with 100 students. I assume that the heights of these students follow a Normal Distribution. I then measure the height of all students, calculate the average height (e.g. "a") and the standard deviation (e.g. "b"). I now have a Normal Distribution(a,b).
If I simulate many samples from this Normal(a,b) - the average value of all these simulations should equal to "a". The way I see it, simulating numbers from this Normal(a,b) has not really provided me with any new information .
This is where my confusion is:
- I can understand why its important to simulate data from a different model to test the robustness of your own model. As an example, if I design a bridge that I believe is well suited to design weights of "x KG" and windspeeds of "y KM/h" - I might be interested in seeing how this bridge behaves under different conditions, such as with weights of " x + z KG" and windspeeds of "y + w KM/h".
- But why is it important to simulate data from the same regression model that you just fit - how can this result any new information that the you can't analytically infer from the model itself?
To reiterate my confusion one more time - I can simulate 1000 random samples from a Normal(a,b) and find out that the average value of these samples is close to "a".... but I could have just taken the Expected value of Normal(a,b) and determined that the average of many samples from this distribution should be close to "a". Thus, what is the importance of simulating from the same model itself?
- Additional Reference: "When conducting simulation studies to evaluate the performance of new and existing statistical methods for analyzing survival data, one is required to simulate event times under a known data generating model. Similarly, one may need to simulate event times for the purpose of power calculations when designing new studies." (https://www.jstatsoft.org/article/view/v097i03)
- However, when reading this quotation - it is not immediately clear to me why simulating data from the same model is required for goals such as "power calculations"