Adjustments to (Linear Regression) Forecast

Question

Full disclosure: I am not a statistician, nor do I claim to be one. I am a lowly IT administrator. Please play gentle with me. :)

I am responsible for collecting and forecasting disk storage use for our enterprise. We collect our storage use monthly and use a simple rolling twelve month linear regression for forecasts (in other words, only the previous twelve months of data are considered when making a projection). We use this information for allocation and capital expense planning, e.g. "Based on this model, we will need to purchase x amount if storage in y months to meet our needs." This all works well enough to suit our needs.

Periodically, we have large one-time movements in our numbers that throws the forecasting off. For example, someone finds 500GB of old backups that aren't needed anymore and deletes them. Good for them for reclaiming the space! However our forecasts are now skewed way off by this large drop in one month. We have always just accepted that a drop like this takes 9-10 months to make its way out of the models, but that can be a really long time if we are entering capital expense planning season.

I'm wondering if there is a way to handle these one-time variances such that the forecasted values aren't impacted as much (e.g. the slope of the line doesn't change as dramatically), but they are taken into account (e.g. a one-time change in the y-value associated with a particular point in time). Our first attempts at tackling this have yielded some ugly results (e.g. exponential growth curves). We do all of our processing in SQL Server if that matters.

Answer 1

What you're looking at are outliers. If you have reason to believe the outlier(s) do not represent your data, you may remove them. In a validated environment, you would have to investigate each one and justify them, but in your case you can probably just delete them.

If you're looking to find these data points automatically, look at Cook's Distance, which analyzes the residuals and can make a mathematical determination of reject criteria (typically 4/n, where n is 12 in your case)

Another suggestion is to open up the data range of your data, can you look at two years, or are data that old completely irrelevant? That of course would reduce the impact of an outlier or two, and also gives more power to analysis methods such as Cook's Distance.

Now the tricky thing can be an offset - so if that outlier causes the entire line to jump down, it will cause the regression to face down even if there's a general upward trend. To prevent that, you can plot the change in hard drive space. Removing the outlier removes the spurious data points, and you can see the overall trend in change in hard drive space, leading to more accurate conclusions.

Answer 2

Here's what I understand of your situation: You have a forecast, a regression model that you evaluate monthly, of your storage needs for "y" months, that uses the data over the previous year from the current month. Once and awhile, someone deletes a chunk of data and suddenly the slope of the line changes dramatically from the usual forecast. This change in the slope effects capital expense planning for however long it takes for the point to run its course through the model.

Your resistance to throwing out the data is appropriate. You have decisions to make a-priori. How to define and handle outliers. Outliers can be defined based on business and/or statistical definitions. I am going to make the assumption that the outliers you are concerned about are simply "obvious".

Once found, we investigate outliers to see if they are generated from a real data generating process, then we handle them according to our a-priori decided upon procedures. These procedures can be anything from 'trimming' a certain percentage of data points from one or both edges of data, to replacing the value, to controlling for the specific outliers, changing the model so it uses a different underlying distribution, or changing the model so it uses a different central tendency.

One way to handle the outliers after they are investigated, assuming the outliers are relatively infrequent, is to add them as a predictor in the model. Each outlier gets an entire predictor. The predictor would be a seperate column that would predict one for the data point that is the outlier and zero otherwise. Once controlled for, they will not pull the slope of the model anymore. This procedure has several advantages, one being that no data are thrown out or changed to some other value.

Some disadvantage include taking extra time to model the outlier and having to re-specify the model each month as the column of the outlier moves through the modeling year window. However, any action with outliers will require extra time in modeling, which is a necessary step in cleaning the data. Also, if I understand correctly, you're respecifying the forecast each month anyway.

Example Regression formula:
    memory_capital = ... + mem_usage_nov + mem_usage_dec + outlier_column1_dec + ...

Answer 3

Here's a simple suggestion. I don't know whether it works for you and maybe I should have made it as a comment, but it seems you need more privileges to make a comment than to make a reply.

If I understand correctly, the figures you are using are the amounts of storage you are using each month. Probably these usualy increase, and you want to predict what the amount will be at some time in the future if trends continue. Once you realise that your big change has happened (e.g. that 500 GB has been released) can you go back and change the previous months' figures (e.g. delete 500 GB from all of them)? Basically what you would be doing is to adjust the previous months' figures to what they should have been, if you knew then what you know now.

Of course I don't recommend this unless you make sure you can go back to the old figures. But the forecasting you want to do sounds like it could even be done in Excel, in which case you can have as many versions as you want.