0
votes
0answers
54 views

Optimism bias - estimates of prediction error

The book Elements of Statistical Learning (available in PDF online) discusses the optimisim bias (7.21, page 229). It states that the optimism bias is the difference between the training error and the ...
2
votes
2answers
746 views

What are acceptable validation or cross validation error rates?

Is there a commonly acceptable error rate for validation? As in, if the error rate is less than X %, then my machine learning method would be considered "successful". I'm looking for something ...
5
votes
1answer
255 views

Name of mean absolute error analogue to Brier score?

Yesterday's question Determine accuracy of model which estimates probability of event got me curious about probability scoring. The Brier score $$\frac{1}{N}\sum\limits _{i=1}^{N}(prediction_i - ...