0
$\begingroup$

I want to compare how well a simulated curve approximates the "real" curve measured on empirical data.

More in detail: I have empirical data, let's say for simplicity the worldwide population per year for the last 200 years. I measure a curve that is mostly increasing with 1 data point for each year. Now I also have a model with some parameters that I use to simulate the same curve. For the simulation I only use the starting point (population at the first year) as well as some universal parameters that I can estimate from any time span in the empirical data and that don't change in time. So I end up with two curves, the real one and the simulated one. Now I want to quantify how accurate my simulation is compared to the empirical data.

How would I approach this apart from plotting both curves on top of each other? Are measures like Mean absolute percentage error or scaled errors the right way?

As I am very new to this topic I apologize if this is perhaps a very trivial or stupid question

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

This is a very very good question. I asked this question myself. This problem is equivalent to logistic regression for instance when you calculate the residuals and sum them up.

So because you are discrete for both the curves you can just calculate: $MSE = \frac {1}{n}\sum_{poins} (y_i-y_j)^2$ where $i$ and $j$ denote first and second curve.

Note: sum is for all points.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your answer! I see how the MSE could provide some measure of quality here, but I am uncertain if it's the right measure for a curve that is increasing almost exponentially. And given a value of MSE for a particular set of curves, how do you judge if the forecast is good or not? $\endgroup$
    – Nik
    Commented Mar 10, 2021 at 17:49
  • $\begingroup$ @Nik these are all good questions, I can say if you have a minimization problem (multiple simulated curves) this method should work, other that that I confirm I don't have the experience to suggest you further solution, this is why other answer are welcome. We all have to learn! $\endgroup$
    – Good Luck
    Commented Mar 10, 2021 at 18:03
  • $\begingroup$ MAPE is also used a good deal in part because it is very simple and easy to interpret. When I reviewed a range of options several years ago the MAPE was the one I found most recommended (although obviously people vary in their opinions). It does not address the questions Nik raises at all, however. $\endgroup$
    – user54285
    Commented Mar 16, 2021 at 22:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.