5
$\begingroup$

In my endeavor to learn forecasting and improve my statistical knowledge, I've decided to forecast the population of a certain area. I've attached the excel file that I used. I know I'm using excel which is not very common around here. I'm familiar with R but prefer excel for fast analysis such as this.

The collected data is for 1921 to 2009. The forecast period is between 2010 and 2019. I did:

  • Arima over all the data
  • Arima over the last 10 years or so of the data
  • Linear regression over the whole time period
  • Linear regression over the last 10 years
  • Fitted model using Eureqa software
  • Average of above models without the overall linear regression.

The forecast worksheet includes all the raw data, while the forecasting simplified worksheet includes the simplified chart to better view the results.

The question, what are some of the ways I can validate and/or calibrate my forecasts, numbers, assumptions, and improve my analysis from an amateur level to something that is more professional.

Thanks

Excel File

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ Glad to see Eureqa being used! $\endgroup$
    – John Doucette
    Commented Oct 19, 2011 at 18:20
  • 1
    $\begingroup$ @JohnDoucette I love Eurqa for fitting models. Also it's a great tool to compare models and find trends at a very fast rate. $\endgroup$
    – dassouki
    Commented Oct 19, 2011 at 18:45
  • 1
    $\begingroup$ Well, I don't think there is something wrong with your question, it is rather a problem with the community -- it is still small and thus even a bounty does not guarantee an answer. You can at most try to break it into a series of smaller, more focused questions (yet I admit it may be hard in this case). Plus a mod can refund a still active bounty. $\endgroup$
    – user88
    Commented Oct 23, 2011 at 14:56
  • $\begingroup$ Good paper on Forecast Validity tests here $\endgroup$
    – user13925
    Commented Sep 8, 2012 at 1:29

1 Answer 1

Reset to default
1
$\begingroup$

Quick three tips after reviewing your excel sheet:

1) If you use a derivative on your metric (population in this case) and try to fit the models through it, you will get a smoother fit as it will eliminate the initial 'jump' in your forecasts. For example, in the "FORECASTS" sheet create a new metric which calculates ((B3/B2)-1) and fit your model through the new metric.

2) Try to find patterns in the data before choosing your period. For example, I see you chose the last 10 years for the prediction model. That is great since the growth significantly slowed down around these years, so probably better for prediction.

3) This is more of a goal based approach. Try to determine what is this forecast going to be used for. If it was for planning food supply for a city, perhaps you prefer to overestimate. If its for tax revenues, you would prefer to underestimate, etc. This will also help you choose the appropriate forecast period, and build a 'range' of forecasts instead of a particular expectation.

Hope this is helpful! And good luck with your work, Erad

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks that was very good feedback. So overall, there is no real validation/calibration for models. What type of errors do you think I should use? When comparing my 2010 and 2011 forecasts to actual 2010 and 2011 numbers to my ARIMA, I was off by 0.49% is that good or bad? How can I measure the error and decide if it was good or not $\endgroup$
    – dassouki
    Commented Oct 28, 2011 at 11:26
  • 2
    $\begingroup$ I'm going to view your question from a practical stand-point. For decision and comparison between ways to measure errors and residuals, you can refer to this wikipedia post. In practice, you need to understand your 'penalty' for each deviation from forecast, up or down. Your error sounds very low, so pretty close to forecast. You should also consider using additional variables of underlying factors to improve the forecast. Perhaps the city population is influenced by unemployment rates, number of schools and hospitals, etc. $\endgroup$
    – Erad
    Commented Oct 31, 2011 at 6:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.