Deriving risk estimates using forecasting confidence limits and out of sample hold-out cases I was hoping for some advice. I use SAS for automatic forecasting (I have a large number of forecasts to complete in a limited timeframe). 
As part of the forecast output from SAS, I get a mid-point (median or mean), and an upper and lower confidence limit for each forecast. This is determined at a pre-specified level (i.e. 95%). Evidently, the confidence limits are derived statistically. I appreciate that within the upper and lower limits there is a range of potential forecasting results that could occur.  
Based on my interpretation, values close to the upper and lower limits of the forecasts will be less likely because they depart a fair way from the mean/median and are close to the tails in terms of the distribution of possible forecasted values. 
In a number of potential forecasting scenarios I face each month, I have a target the business needs to achieve by the end of the next financial year (june 30, 2012). I also have a forecast of the likely future value + upper and lower forecast limits for end of year (june 30, 2012) derived using moderate to long term historical performance. I need to quantify the probability of reaching a target given a forecasted result, using the point where the target is located relative to the mean/median and also the upper and lower forecast limits.  For example, if the series has a mean forecast of 50, a LCL of 0 and UCL of 100 and the target is 75, I need to quantify the probability this target will be met at June 30, 2012. 
It seems to me I can use the fact the target of 75 falls on the 75th percentile relative to the forecast upper and lower forecast limits. 


*

*Is it fair to say that there is a 25% chance of hitting the target under an assumed uniform distribution? 

*Or is it more appropriate to assume the forecast series within the upper and lower limits is normally distributed with a greater concentration of values in the CDF closer to the mean?

*Or is this model dependent (i.e. different for arima and esm models)


Also, as a rule of thumb how many holdouts do people use for 


*

*a series with 12 months (I am using 10% or 1)

*a series with 24 months (I am using 10% or 2)

 A: In theory the confidence intervals are derived from the estimated distribution of the forecast. This means that the model gives estimate $\hat{y}$ and its estimated cdf $\hat{F}$. The 95%-confidence interval is then calculated as 
$$(\hat{F}^{-1}(0.025),\hat{F}^{-1}(0.975))$$
If $\hat{F}$ would be available you could assign (with certain assumptions) probability for the target $y$ by simply reporting $1-\hat{F}(y)$ which will be the probability $P(\hat{y}>y)$, i.e. the probability that the forecast will be larger than the target. 
However you do not have $\hat{F}$, you just have 2 values: $\hat{F}^{-1}(0.025)$ and $\hat{F}^{-1}(0.975)$. In general this is too little information to recover $\hat{F}$. On the other hand it helps to know that for a lot of models $\hat{F}$ is normal distribution. Usually:
$$\hat{y}\sim N(\mu_{\hat y},\sigma^2)$$
where $\mu_{\hat y}$ is the reported mean or median forecast. Since in this case the 95% confidence interval is 
$$(\mu_{\hat y}-1.96\sigma,\mu_{\hat y}+1.96\sigma)$$
It is not hard then to recover $\sigma$ and for the target $y$ we have
$$P(\hat{y}>y)=1-\Phi\left(\frac{y-\mu_{\hat y}}{\sigma}\right)$$
where $\Phi$ is standard normal distribution. 
This should be used with caution, since although a lot of models do use normal distribution for $\hat{F}$, this is not always the case. Furthermore it might be that for example $\log y$ was modelled, but confidence interval was reported for $y$. 
So to sum up the answers to your questions would be:


*

*You can say that, but the uniform distribution assumption is almost guaranteed to be false.

*Yes you can suppose that with some confidence, but you need to check the model details to be sure.

*Yes it is model dependent. As I illustrated you can recover something, since a lot of models use the same approach in calculating confidence intervals, but there is a lot of models too where some different approach is used.

