Confidence and Prediction Intervals and Cross-Validation So suppose, I have 5000 points of data. I hold out 1000 for validation testing, and conduct simple linear regression on the first 4000 points. 
How can I determine if the linear model: y = a + bx + e, reasonably explains the relationship between x and y using this technique? 
I've heard that prediction and confidence intervals are very useful in these situations, but am not too sure how to apply them.
 A: "How can I determine whether a linear regression is good" is quite a long question. You also mention training and test sets which is in many ways a separate question. 
When checking a linear regression you typically do what we call "regression diagnostics". You can look up a lot more online about this and probably find some straight-forward videos. And example link that is a bit dense is: http://people.duke.edu/~rnau/testing.htm. Different people have a different art to determining whether the model meets the assumptions. R-squared is often looked at, normality plots are often looked at (and non-normality may drive larger confidence intervals), etc. Your goal here is to determine whether the requirements of linear regression are met and fix them if they aren't.
Confidence intervals and prediction intervals will give you an idea of how far off your model will be from reality. How close you need your confidence bands depends on your requirements. If you're building the space shuttle you probably need to have a tighter tolerance on component manufacturing than you do if you're building a child's toy firetruck. 
If your linear regression matches all the assumptions required for linear regression, and your diagnostic shows a good regression, then you can get ready to use your test set. Your test set is used to validate whether your model is over-fit to the training set. If your training set doesn't represent the total population then your test set will have significant differences in performance than your training set did. This shows you that you have a problem with the model or that you don't have enough data about the total population being input into your model (or some other problem). 
