2
$\begingroup$

I am very new at machine learning, and I'm building an artificial neural network that aims to classify inputs into 2 labels. I am training the network with randomly initialized weights and through the stochastic gradient descent. What would you say is an acceptable accuracy for training set and test set? Is around 80% for test set good enough or should I aim to get close to 100%?

Thank you so much in advance.

Improve this question
$\endgroup$
2
  • $\begingroup$ Acceptable for what purpose? What are the costs for incorrect classification? We can't tell you what the best choice is because we don't know about your context, rewards, costs and constraints. $\endgroup$
    – Sycorax ♦
    May 22, 2020 at 12:53
  • $\begingroup$ You are right, I now see that. For the purpose of classification, the cost isn't very high for a wrong classification, because the aim is only to aid in decision making and not to make final decisions. Thank you so much for replying. Best wishes $\endgroup$
    – Johanna
    May 22, 2020 at 14:57

1 Answer 1

Reset to default
1
$\begingroup$

It’s impossible to say in general. It’s even impossible to say in general what makes for an acceptable $R^2$ value in linear regression! This will depend on the problem at hand and what has been done on the same or similar data sets. You may think that 95% accuracy is fantastic. For MNIST, for instance, that’s nothing special.

Finally, I would encourage you to read what Frank Harrell and others on this Stack have to say about accuracy (even AUC) as opposed to “proper scoring rules” (such as Brier score). Shamelessly, I will link a post of mine where I challenge this a bit, though it even contains an example where you can see an argument for why accuracy might not be the best performance metric.

Proper scoring rule when there is a decision to make (e.g. spam vs ham email)

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you so much Dave. Your post in the link did help to clarify this matter, that is indeed a difficult question. I have been thinking and I think I found what would be an acceptable accuracy for my problem :) Best wishes $\endgroup$
    – Johanna
    May 22, 2020 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.