I often hear about evaluating a classification model's performance by holding out the test set and training a model on the training set. Then creating 2 vectors, one for the predicted values and one for the true values. Obviously doing a comparison allows one to judge the performance of the model by its predictive power using things like F-Score, Kappa Statistic, Precision & Recall, ROC curves etc.
How does this compare to evaluating numeric prediction like regression? I would assume that you could train the regression model on the training set, use it to predict values, then compare these predicted values to the true values sitting in the test set. Obviously the measures of performance would have to be different since this isn't a classification task. The usual residuals and $R^2$ statistics are obvious measures but are there more/better ways to evaluate the performance for regression models? It seems like classification has so many options but regression is left to $R^2$ and residuals.