0
$\begingroup$

If the activations of different layers of a neural network are perturbed, will the network be able to recover from that and do the correct prediction? Is there a training method that makes neural networks resilient to noise?

$\endgroup$

1 Answer 1

3
$\begingroup$

If the activations of different layers of a neural network are perturbed, will the network be able to recover from that and do the correct prediction?

That depends on the amount of noise, and if the network was trained with this noise. If you train network without any noise, it won't be able to handle it later. Also, if the noise is too strong, the network will probably give you bad results.

Is there a training method that makes neural networks resilient to noise?

There are plenty. Take dropout for example: Dropout trained networks are by default made resilient to multiplicative binary noise. Another example are noisy autoencoders [1].


[1]: Poole B. et al., 2014: Analyzing noise in autoencoders and deep networks.

$\endgroup$
1
  • $\begingroup$ In the context of quantized activations, e.g. 16 different values for each activation, is there a method that deals with a noise where some of these activations are randomly altered to one of these quantized values? $\endgroup$
    – Matt
    Commented Sep 25, 2018 at 23:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.