a measure for MAE (of a regression) I'm running a grid search, in order to fine-tune a NN hyper parameters.
the question is:
the MAE values I get from the trainings are too close.
since I have the statistical attributes of the target values, is there a way to somehow come up with a starting value for MAE, where the worst regression can achieve.
I think the question is not clear.
the value I'm looking for, is analogous to a probability for classification problems. (example: an neural network which is supposed to classify the inputs into 8 possible classes, will have accuracy of 0.125 just by random classification of inputs. so a 12.5% accuracy is the measure for such classifier)
now, I have the Mean, and StdDev (and other stats if need be) of the target values of my samples. how can I calculate a measure to judge my MAE (and not by comparing different MAE values from different trainings)?
 A: I think what you’re looking for is the median of the pooled data.
In a classification problem, the most naive way to find class probabilities is to use your training data. In MNIST, you have a 10% chance of each number. If you’re a doctor specializing in disease X than affects 7% of people, say that everyone has a 7% chance of having disease X.
The point of regression is to tighten up that estimate for a subject with some other values measured. Perhaps the doctor knows that 7% of people have the disease but that 70% of people who work around asbestos have the disease. If a patient works around asbestos, then 7% probably is the wrong probability to assume. Maybe 70% is wrong, but it’s probably more along the lines of the right answer than 7%. (Or maybe not. Maybe there’s some other covariate like smoking status or genetics that should be included in the model.)
When you do a regression with a numeric  response variable, the naive way to guess the conditional parameter of interest (often the conditional mean, given covariates like asbestos), is to guess the estimate from the pooled distribution of all training data. This is why r-squared compares to the sum of deviations from the mean of the response variable, as that would be the most naive sensible way to guess. If your regression predicts median instead of mean, then your first guess would be the median of the pooled data. Ditto for other quantile regressions (e.g. 0.25 quantile). 
