How to evaluate results of linear regression

Question

I have a linear regression problem. In short, I have a dataset, I divided it into two subsets. One subset is used to find the linear regression (training subset), another is used to evaluate it (evaluation subset). My question is how to evaluate the result of this linear regression after applying it to the evaluation subset of data?

Here are the details:

In the training subset, I do linear regression: $y = ax + b$, where $y$ is groundtruth (also known as target), $x$ is an independent variable. Then I found $a$ and $b$. ($x$ and $y$ are given in the training subset).

Now, using $a$ and $b$ found above from the training subset, apply them to the evaluation subset, I found $y' = ax' + b$. In other words, these $y'$ are found from linear regression with $x'$. Now, in addition to $y'$, I also have $y$ from the evaluation set. How do I evaluate my result (how much $y'$ differ from $y$)? Any general mathematical model to do that? It needs to be some sort of math model/formula. I can think of different ways to do it, but they are all kinda ad-hoc or simple, but this is for a scientific work, so things that sound ad-hoc cannot be used here, unfortunately.

Any idea?

Answer 1

I'd agree with @Octern that one rarely sees people using train/test splits (or even things like cross-validation) for linear models. Overfitting is (almost) certainly not an issue with a very simple model like this one.

If you wanted to get a sense for your model's "quality", you may want to report confidence intervals (or their Bayesian equivalents) around your regression coefficients. There are several ways to do this. If you know/can assume that your errors are normally distributed, there's a simple formula (and most popular data analysis packages will give you these values). Another popular alternative is to compute them through resampling (e.g., bootstrapping or jackknifing), which makes fewer assumptions about the distribution of errors. In either case, I'd use the complete data set for the computation.

Answer 2

if you really are fine with your linear trainig model and want to know how well it would predict your test data, then all you would have to do is to use the linear model formula you already have and include the estimated coefficients a (= intercept) and b(regression coefficient, also called slope) resulting from first model.

should look like y= a + b*X here some imaginary numbers... y= 2 + 0.5*X

Which software are you using? Are you using R ? if so, you can use the predict.lm() function and apply it on your 2nd dataset.

Answer 3

While this largely depends on exactly what your goals are, a simple and standard way to do this would be measuring the mean squared error (MSE). So if you have your test dataset $\mathcal{D}$ which consist of input/output pairs, $\mathcal{D} = \{(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)\}$ and your parameters $a$ and $b$, then the MSE can be calculated as

$$ \text{MSE}_{a,b} = \frac{1}{n}\sum_{i=1}^n (y_i - (ax_i + b))^2. $$

This is probably a sensible way to measure your error also since this is likely the criteria you used for finding the parameters $a$ and $b$. If you want to get a better idea of how well your estimated parameters generalize, you should look into something like cross validation.