Assessing forecastability of time series Suppose i have a little over 20.000 monthly time series spanning from Jan'05 to Dec'11.
Each of these representing global sales data for a different product. What if, instead of computing forecasts for each and every one of them, I wanted to focus only on a small number of products that "actually matter"?
I could rank those products by total annual revenue and trim down the list using classical Pareto. Still it seems to me that, although they do not contribute much to bottom line, some products are so easy to forecast that leaving them out would be bad judjement. A product that sold 50$ worth each month for the past 10 years might not sound like much, but it requires so little effort to generate predictions about future sales that I might as well do it.
So let's say I divide my products in four categories: high revenue/easy to forecast - low revenue/easy to forecast - high revenue/hard to forecast - low revenue/hard to forecast.
I think it would be reasonable to leave behind only those time series belonging to the fourth group. But how exactly can I evaluate "forecastability"?
Coefficient of variation seems like a good starting point (I also remember seeing some paper about it a while ago). But what if my time series exhibit seasonality/level shifts/calendar effects/strong trends?
I would imagine I should base my evaluation only on variability of the random component and not the one of the "raw" data. Or am I missing something?
Has anybody stumbled upon a similar problem before? How would you guys go about it?
As always, any help is greatly appreciated!
 A: This is a fairly common problem in forecasting. The traditional solution is to compute mean absolute percentage errors (MAPEs) on each item. The lower the MAPE, the more easily forecasted is the item.
One problem with that is many series contain zero values and then MAPE is undefined.
I proposed a solution in Hyndman and Koehler (IJF 2006) [Preprint version] using mean absolute scaled errors (MASEs). For monthly time series, the scaling would be based on in-sample seasonal naive forecasts. That is if $y_t$ is an observation at time $t$, data are available from times 1 to $T$ and
$$
Q = \frac{1}{T-12}\sum_{t=13}^T |y_t-y_{t-12}|,
$$
then a scaled error is $q_t = (y_t-\hat{y}_t)/Q$, where $\hat{y}_t$ is a forecast of $y_t$ using whatever forecasting method you are implementing for that item. Take the mean absolute value of the scaled errors to get the MASE. For example, you might use a rolling origin (aka time series cross-validation) and take the mean absolute value of the resulting one-step (or $h$-step) errors.
Series that are easy to forecast should have low values of MASE. Here "easy to forecast" is interpreted relative to the seasonal naive forecast. In some circumstances, it may make more sense to use an alternative base measure to scale the results.
A: You might be interested in ForeCA: Forecastable Component Analysis (disclaimer: I am the author). As the name suggests it is a dimension reduction / blind source separation (BSS) technique to find most forecastable signals from many multivariate - more or less stationary - time series. For your particular case of 20,000 time series it might not be the fastest thing to do (the solution involves multivariate power spectra and iterative, analytic updating of the best weightvector; furthermore I guess it might run into the $p \gg n$ problem.)
There is also an R package ForeCA available at CRAN (again: I am the author) which implements basic functionality; right now it supports the functionality to estimate forecastability measure $\Omega(x_t)$ for univariate time series and it has some good wrapper functions for multivariate spectra (again 20,000 time series is probably too much to handle at once).
But maybe you can try to use the MASE measure proposed by Rob to make a coarse grid separation of the 20,000 in several sub-groups and then apply ForeCA to each separately.
A: Here's a second idea based on stl. 
You could fit an stl decomposition to each series, and then compare the standard error of the remainder component to the mean of the original data ignoring any partial years. Series that are easy to forecast should have a small ratio of se(remainder) to mean(data).
The reason I suggest ignoring partial years is that seasonality will affect the mean of the data otherwise. In the example in the question, all series have seven complete years, so it is not an issue. But if the series extended part way into 2012, I suggest the mean is computed only up to the end of 2011 to avoid seasonal contamination of the mean.
This idea assumes that mean(data) makes sense -- that is that the data are mean stationary (apart from seasonality). It probably wouldn't work well for data with strong trends or unit roots.
It also assumes that a good stl fit translates into good forecasts, but I can't think of an example where that wouldn't be true so it is probably an ok assumption.
A: This answer is very late, but for those who are still looking for an appropriate measure of forecastability for product demand time series, I highly suggest looking at approximate entropy. 

The presence of repetitive patterns of fluctuation in a time series renders it more predictable than a time series in which such patterns are absent. ApEn reflects the likelihood that similar patterns of observations will not be followed by additional similar observations.[7] A time series containing many repetitive patterns has a relatively small ApEn; a less predictable process has a higher ApEn.

Product demand tend to have a very strong seasonal component, making the coefficient of variation (CV) inappropriate. ApEn(m, r) is able to correctly handle this. In my case, since my data tends to have a strong weekly seasonality, I set parameters m=7 and r=0.2*std as recommended here. 
