What are the various techniques and metrics used to evaluate how accurate / good an algorithm is? How would you use a given metric to derive a conclusion about a ML model?

  • $\begingroup$ There are no "metrics" to evaluate machine learning algorithms since machine learning algorithms do not constitute a metric space, I think you mean what "measures of performance are there". $\endgroup$
    – samthebest
    Jun 1 '15 at 14:09
  • $\begingroup$ I have summarized all the important metrics used in machine learning in this article medium.com/machine-learning-through-visuals/… $\endgroup$ Aug 15 '18 at 20:17

Some possible methods:

  1. Focus on the predictive capability of a model: confusion matrix (computing the accuracy, F1-score, cost of classification). You can draw an ROC curve, and performance of every classifier is represented as a point on the curve. When you change the threshold in the algorithm, sample distribution, or the cost matrix of classification, the point locations will change as well.

  2. Learning curve (bias-variance tradeoff). It helps determine the sample size and the feature size.

  3. Calculate the ratio between the predictive accuracy of the model and the baseline accuracy (without using your model). It is also called lift chart.

  4. Confidence interval of accuracy.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.