In Intriguing properties of neural networks (https://arxiv.org/pdf/1312.6199.pdf) they show (4.3), that the existance of adversarial examples is closely connected to the upper Lipschitz constant, namely that if the constant is small enough then it's not possible to find adversarial examples. As it turns out these Lipschitz constants are bounded by respective norms of weight matrices (in case of fully connected network). Has anyone tried to regularize network with respect to these Lipschitz constants or matrix norms, so that the network is fully resistant to adversarial examples? If not, then why? Is it too complex when it comes to computations, not very efficient, or something else?

  • $\begingroup$ This line in the conclusion drew my attention: "The existence of the adversarial negatives appears to be in contradiction with the network’s ability to achieve high generalization performance." Perhaps you can't regularize the network wrt to Lipshitz constants AND get good generalization performance. I hope I am wrong here. $\endgroup$ – Vladislavs Dovgalecs Dec 18 '17 at 21:55
  • $\begingroup$ Oh, I totally overlooked that sentence, thanks. It might be the case that such a strong regularization might cause the network to behave strangely and not generalize well. Anyway, I'm going to leave this question open for now, maybe someone else will give us some more insight. $\endgroup$ – G.Fil Dec 18 '17 at 21:59
  • 2
    $\begingroup$ Here's an example of such regularization: arxiv.org/pdf/1704.08847.pdf $\endgroup$ – Alex R. Dec 18 '17 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.