Return to Answer

I would first try some basic tests to ascertain if your methodology is producing any forecast of value.

For example, assume that there is a possible interest in a correct forecast greater (or less) than say k%. Tabulate the number of times the database indicated a forecast value greater than k% in total of n cases. Compare this to the actually observed times it occurred. Use a statistical test to assign significance (like, for example, a Pearson's Chi-Square test).

Repeat for different potential values of interest and record statistical test results.

How is your model performing? You now should be better able to answer the question: "When would one say the fitted model is NOT a good fit?" Also, you now have an intuitive "goodness-of-fit measures that (somehow) take into account the already measured uncertainties".

I would first try some basic tests to ascertain if your methodology is producing any forecast of value.

For example, assume that there is a possible interest in a correct forecast greater (or less) than say k%. Tabulate the number of times the database indicated an expected forecast value greater than k% in total of n forecast cases. Compare this to the actually observed times it correctly occurred. Use a statistical test to assign significance (like, for example, a Pearson's Chi-Square test).

Repeat for different potential values of interest and record statistical test results.

How is your model performing? You now should be better able to answer the question: "When would one say the fitted model is NOT a good fit?" Also, you now have an intuitive "goodness-of-fit measures that (somehow) take into account the already measured uncertainties".

I would first try some basic tests to ascertain if your methodology is producing any forecast of value.

For example, assume that there is a possible interest in a correct forecast greater (or less) than say k%. Tabulate the number of times the database indicated a forecast value greater than k% in total of n cases. Compare this to the actually observed times it occurred. Use a statistical test to assign significance (like a Pearson's Chi-Square test).

Repeat for different potential values of interest and record statistical test results.

How is your model performing? You now should be better able to answer the question: "When would one say the fitted model is NOT a good fit?"

I would first try some basic tests to ascertain if your methodology is producing any forecast of value.

For example, assume that there is a possible interest in a correct forecast greater (or less) than say k%. Tabulate the number of times the database indicated a forecast value greater than k% in total of n cases. Compare this to the actually observed times it occurred. Use a statistical test to assign significance (like, for example, a Pearson's Chi-Square test).

Repeat for different potential values of interest and record statistical test results.

How is your model performing? You now should be better able to answer the question: "When would one say the fitted model is NOT a good fit?" Also, you now have an intuitive "goodness-of-fit measures that (somehow) take into account the already measured uncertainties".

I would first try some basic tests to ascertain if your methodology is producing any forecast of value.

For example, assume that there is a possible interest in a correct forecast greater (or less) than say k%. Tabulate the number of times the database indicated a forecast value greater than k% in total of n cases. Compare this to the actually observed times it occurred. Use a statistical test to assign significance (like a Pearson's chi-square test).

Repeat for different potential values of interest and record statistical test results.

How is your model performing?

I would first try some basic tests to ascertain if your methodology is producing any forecast of value.

For example, assume that there is a possible interest in a correct forecast greater (or less) than say k%. Tabulate the number of times the database indicated a forecast value greater than k% in total of n cases. Compare this to the actually observed times it occurred. Use a statistical test to assign significance (like a Pearson's Chi-Square test).

Repeat for different potential values of interest and record statistical test results.

How is your model performing? You now should be better able to answer the question: "When would one say the fitted model is NOT a good fit?"