What’s the most appropriate statistical analysis to compare predictions to observations? I have a small set (say, 200 data points) of estimates about the time to complete a varied group of tasks, along with the actual times used to complete those tasks. 
These estimates are divided in four groups: 0:30, 1:00, 2:00, and 4:00. These groups indicate that the task should be completed in 30 minutes, 1 hour, and so on.
The hypotheses that I’d want to test are:


*

*What is the correlation, if any, between the estimate and the difference with the actual times? In other words, are the smaller groups more precise and/or more reliable?

*Are the times significantly under- or overestimated? 


PS
I don’t have a solid knowledge of statistics beyond the really really basic stuff. Maybe my question is meaningless, I don’t know. If that’s the case, sorry for the noise.
 A: I'm not entirely sure how to answer your question, so I'll just throw out a couple of thoughts here, and maybe something will help.  It occurs to me that it depends where estimates came from.  For example, people will often use a basic multiple regression model to calculate predictions for some outcome variable based on a set of known factors.  At the end of this process, people properly want to know how well the model's predictions do.  


*

*A basic, common approach is to simply correlate the predictions with the observed values.  Obviously, you want as strong a positive correlation as possible.  Moreover, if you square the correlation, that gives you the common metric $R^2$, a measure of the model's informativeness.  

*As for the correlation between the predictions and the difference
between the prediction and the observed value, you want it to be 0,
and it should be when coming from a multiple regression model.  Should it not be 0, that implies a bias in the process that generated the predictions.  

*Your sample size (~200) should be adequate so long as you don't have
too many predictor variables.  In experiments (where observations are assigned to levels of the factors such that they are orthogonal) a typical recommendation is to have at least 10 observations per factor.  With observational research, you'll want more, depending on the level of inter-correlation amongst the predictor variables.  

*As for the idea of predictions being 'significantly different' from responses, that depends on the model.  In linear regression, it implies that the model is misspecified (e.g. fitting curvilinear data with a straight line).  

*One last note, it is known that the variance of a time increases with
mean time.  In other words, although something takes usually an hour to finish,
sometimes it will take more or less.  Likewise for a task that
typically takes two hours, but in the latter case, the more or less
will be spread out further than in the former case.  You should check
for this in your data.  If you find it, you may want to transform
your completion times by taking the logarithm of the values, and then recalculate your predictions.  


I can't tell if the situation you face is the same as a common multiple regression situation, but ideas taken from there may be of help.  Let me know if the information you need is different from this.
