Measuring forecasting risk of linear regression

Question

I want to measure how much risk I take by forecasting something. I know I can measure the error and things like MAD, MSE, RMSE, etc. and I can set up prediction intervals but I'd like to describe a little more about the 'amount' of risk I take, like, how big will my errors be and how often will they occur. Are there measurements for this which I can compare with a acceptable 'amount' that I defined beforehand?

I have a linear regression model.

Answer 1

1. How big the errors are?

This only requires a simple plot of errors where "errors=|prediction - target|". You can also run max(errors), min(errors) and mean(errors). The "acceptable" amount depends on your model and work, but you could consider the "mean(errors)" as your acceptable amount of errors.

2. How often they occur?

You can try histogram which gives you an overview of distribution of your errors as well.

All these works can be easily done in MATLAB through "Curve-Fitting" Toolbox.