How to interpret error measures? I am running the classify in Weka for a certain dataset and I've noticed that if I'm trying to predict a nominal value the output specifically shows the correctly and incorrectly predicted values. However, now I'm running it for a numerical attribute and the output is:
Correlation coefficient                 0.3305
Mean absolute error                     11.6268
Root mean squared error                 46.8547
Relative absolute error                 89.2645 %
Root relative squared error             94.3886 %
Total Number of Instances               36441 

How do I interpret this? I've tried googling each notion but I don't understand much since statistics is not at all in my field of expertise. I would greatly appreciate an ELI5 type of answer in terms of statistics.
 A: Let's denote the true value of interest as $\theta$ and the value estimated using some algorithm as $\hat{\theta}$. 
Correlation tells you how much $\theta$ and $\hat{\theta}$ are related. It gives values between $-1$ and $1$, where $0$ is no relation, $1$ is very strong, linear relation and $-1$ is an inverse linear relation (i.e. bigger values of $\theta$ indicate smaller values of $\hat{\theta}$, or vice versa). Below you'll find an illustrated example of correlation.

(source: http://www.mathsisfun.com/data/correlation.html)
Mean absolute error is:
$$\mathrm{MAE} = \frac{1}{N} \sum^N_{i=1} | \hat{\theta}_i - \theta_i | $$
Root mean square error is:
$$ \mathrm{RMSE} = \sqrt{ \frac{1}{N} \sum^N_{i=1} \left( \hat{\theta}_i - \theta_i \right)^2 } $$
Relative absolute error:
$$ \mathrm{ RAE} = \frac{ \sum^N_{i=1} | \hat{\theta}_i - \theta_i | } {  \sum^N_{i=1} | \overline{\theta} - \theta_i | } $$
where $\overline{\theta}$ is a mean value of $\theta$.
Root relative squared error:
$$ \mathrm{ RRSE }= \sqrt{ \frac{ \sum^N_{i=1} \left( \hat{\theta}_i - \theta_i \right)^2 } {  \sum^N_{i=1} \left( \overline{\theta} - \theta_i \right)^2 }} $$
As you see, all the statistics compare true values to their estimates, but do it in a slightly different way. They all tell you "how far away" are your estimated values from the true value of $\theta$. Sometimes square roots are used and sometimes absolute values - this is because when using square roots the extreme values have more influence on the result (see Why square the difference instead of taking the absolute value in standard deviation? or on Mathoverflow).
In $ \mathrm{ MAE}$ and $ \mathrm{ RMSE}$ you simply look at the "average difference" between those two values - so you interpret them comparing to the scale of your valiable, (i.e. $ \mathrm{ MSE}$ of 1 point is a difference of 1 point of $\theta$ between $\hat{\theta}$ and $\theta$).
In $ \mathrm{ RAE}$ and $ \mathrm{ RRSE}$ you divide those differences by the variation of $\theta$ so they have a scale from 0 to 1 and if you multiply this value by 100 you get similarity in 0-100 scale (i.e. percentage). The values of $\sum(\overline{\theta} - \theta_i)^2$ or $\sum|\overline{\theta} - \theta_i|$ tell you how much $\theta$ differs from it's mean value - so you could tell that it is about how much $\theta$ differs from itself (compare to variance). Because of that the measures are named "relative" - they give you result related to the scale of $\theta$.
Check also those slides.
