MSE Intuition and Interpretation I've got a very small question. Say I'm making a linear regression model. When I test the model with a testing set, I get an MSE of 4.31 (arbitrary). What do I interpret from this? As in, what does this '4.31' represent, or rather mean?
Until now, I was thinking of it as the average squared difference between target and predicted values. Thus, I thought it showed that the average prediction differed by sqrt(4.31) from the average actual value, but I'm not sure.
 A: 
what does this '4.31' represent, or rather mean?
Until now, I was thinking of it as the average squared difference between target and predicted values.

This is exactly correct.

Thus, I thought it showed that the average prediction differed by sqrt(4.31) from the average actual value, but I'm not sure.

And this is incorrect.
That the mean squared error (MSE) is $4.31$ does not imply that the mean absolute error (MAE) is $\sqrt{4.31}$. In fact, there is no simple relationship between the MSE and the MAE that always holds.

Here is an example. Suppose your actuals are iid standard normally distributed, $y\sim N(0,1)$. You have two different predictions.

*

*The first prediction is a flat zero, $\hat{y}=0$.

*

*The expected MSE of this is the expectation of a $\chi_1$ distribution, which is $1$.

*The expected MAE of this is the expectation of a standard normal distribution left truncated at zero, which is $\frac{2}{\sqrt{2\pi}}\approx 0.80$.



*The second prediction is perfect, $\hat{y}=y$, with probability $0.9$, and off by $\sqrt{10}\approx 3.16$, $\hat{y}=y+\sqrt{10}$ with probability $0.1$.

*

*The expected MSE of this is $0.1\times 10=1$, as above.

*The expected MAE of this is $0.1\times\sqrt{10}\approx0.32$.



We see that you can have the same MSE but different MAEs.
A: I'll just formulate some ideas and give leads to free introductory material.
The raw MSE figure in itself doesn't tell you much. 
Depending on the quantities in your sample space, sqrt(4.31) could be very small, or very big, which may make you want to use the model or not. 
What you can use it for is the comparison of different models, and favor the one with smallest MSE. Within the data you have, your model with MSE = 4.31 could be the worst as well as the best. 
From the Wikipedia Mean squared error page: 

The MSE is a measure of the quality of an estimator—it is always non-negative, and values closer to zero are better. 

The article contains further links about specifics on regression and prediction if you look for more advanced content.
For an accessible video introduction: Khan Academy has a serie on Squared Error
To sum it up: MSE is a relative measure of how well your model fits your sample data.
There is more to it if you know more about the distribution of the different variables, dependent, independent and residuals, and you might be interested in ANOVA and goodness of fit to go further on the topic. 
I hope this helps,
All the best,
Mathieu
