What type of simulation would be needed?

Question

I have come up with 10 different regression models (one for every sector of the S&P500) and was interested in running a simulation as a way of testing the performance of the models. I have a few questions related to this:

1) Is simulation a reasonable approach for determining possible performance? 2) How would I go about building a simulation in R if I want to calculate: average net profit/loss, beta, sharpe ratio, Volatility, Average Gain, Average Loss, win-to-loss ratio, and annualized returns?

Thanks!

Answer 1

1) No, at least when by "simulation" you mean simulating the values of S&P index. It's circular reasoning because in order to simulate you need to know the data generating process and if you know it then you know the true model already.

One possible exception is when you simulate something like a pure random walk and see if your model selection procedure come up with a well performing solution. Since one can't beat a random walk that would imply that your model is overfitted.