I'm new to Neural Networks. Trying to get some general advice.

Multi Class, 3 classes

Has noisy labels, with somewhere between 60 and 80 percent accuracy

Huge amount of training with the issues mentioned

Classes known to follow a roughly 3:2:1 distribution in terms of abundance

Feature Space has the ability to overfit if too much interaction allowed

Features have some noise as well, enough not to easily yield to linear decomposition

I've used neuralnet and now using ANN2, mainly due to having regularization exposed. I've attempted to denoise the labels with an ensemble denoising procedure found in NoiseFiltersR with some success. If it weren't for the noisy labels, what I'm trying to do would have been done a hundred times over.

Would like some advice relating to:

Recommended Neural Network Architecture, especially as it relates to the noisy labels.


1 Answer 1


Noisy labels for neural networks it is a challenging task. As they have a very flexible power of representation they adjust very well to data resulting in overfitting. Overfitting could lead us to misclassifying instances in test. Besides, if we have noisy labels, we will have even a worse model.

I think there is not a closed topic and it is currently an interesting topic in data science research. Specially, in medical topics where the labels are usually noisy (because the difficult of labelling, doubtful cases,...) are many papers with several different approaches.

I recommend you this paper: https://arxiv.org/pdf/1803.11364.pdf They use Expectation-Maximization (EM), learning both clean labels and the model iteratively together. They update weights with fixed labels and them they update labels with fixed weights. They also incorporate other things. Look at the error loss and the regularization terms. In the error loss regularizer $\mathcal{L}_p$ you can incorporate your information about expected label proportion. Besides, the regularizer $\mathcal{L}_e$ forces to predict one clean label and avoid so much uncertainty when predicting clean labels.

In this way, you learn both neural networks weights and clean labels.

  • $\begingroup$ Thanks. I've upped, but too new to count for much. My scholar searches did not turn this one up, a great paper. $\endgroup$
    – Duncan Lee
    Aug 12, 2019 at 23:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.