I'm new to forecasting and I have some (probably very) basic and generic questions. I'd appreciate some references that get into details of this too.
Using some model to forecast a time series, I generate a number of MCMC samples and use the median as the forecast and an interval around it as the prediction interval. My question is:
- What are some standard practices in choosing the interval width. in particular, why not 100%?
- When would one say the fitted model is NOT a good fit? i.e. looking at a particular measure of goodness-of-fit, like $R^2$ or normalized root mean square error, how does one choose a threshold for accepting/rejecting the model?
- Are there any goodness-of-fit measures that (somehow) take into account the already measured uncertainties?
Note 1: for practical reasons I'm not interested in out-of-sample validations.
Note 2: I'm probably asking "wrong" question, please feel free to direct me to the "right" questions I should be asking.