What is bias correction? I have seen many places where they have input/output datasets where they first create a linear regression line, correct the bias, and then only use that data for their model. I didn't get what this bias correction is?
 A: Although the problem statement is not precise enough to know exactly what type of bias correction you are referring to, I think I can talk about it in general term.  Sometimes an estimator can be biased.  This merely means that although it may be a good estimator, its expected or average value is not exactly equal to the parameter. The difference between the estimator's average and the true parameter value is called the bias.  When an estimator is known to be biased, it is sometimes possible, by other means, to estimate the bias and then modify the the estimator by subtracting the estimated bias from the original estimate.  This procedure is called bias correction.  It is done with the intent of improving the estimate.  While it will reduce the bias it will also increase the variance.  So for it to be useful the improvement in bias must be large relative to the loss in the variance.
A good example of successful bias correction is the bootstrap bias correction estimates of classification error rate.  The resubstitution estimate of error rate has a large optimistic bias when the sample size is small.  The bootstrap is used to estimate the bias of the resubstitution estimate and since the resubstitution estimate underestimates the error rate the bias estimate is added to the resubstitution estimate to get the bootstrap bias corrected estimate of the error rate.  When the sample size is small 30 or less combining both classes in a two class problem certain forms of the bootstrap estimate (particularly the 632 estimate) provide more accurate estimates of the error rates than leave-one-out cross validation  (which is a very nearly unbiased estimate of error rate).
