Why does the fast gradient sign method use only the sign of the gradient? In the paper "Explaining and Harnessing Adversarial Examples", Goodfellow introduces the FGSM.
What is the reasoning behind using the sign of the gradients instead of the gradients itself? It feels like you are just throwing out useful information when you just use the sign.
For example, if the gradient for a pixel is 0.00001, FGSM will still perturb that pixel by the same amount as a pixel whose gradient is 10. Why does this make sense? as ideally you would want to perturb pixels in proportion to their gradient.
 A: This is something I have wondered myself, but recently discovered an answer in the original paper Explaining and Harnessing Adversarial Examples:

Because the derivative of the sign function is zero or undefined everywhere, gradient descent on
the adversarial objective function based on the fast gradient sign method does not allow the model
to anticipate how the adversary will react to changes in the parameters. If we instead adversarial
examples based on small rotations or addition of the scaled gradient, then the perturbation process
is itself differentiable and the learning can take the reaction of the adversary into account. However,
we did not find nearly as powerful of a regularizing result from this process, perhaps because these
kinds of adversarial examples are not as difficult to solve.

A: This is answered on page 2 of "Explaining and Harnessing Adversarial Examples" by  Ian J. Goodfellow, Jonathon Shlens & Christian Szegedy.
The purpose of the attack is to find examples from class $i$ that are misclassified as class $j\neq i$ by adding some noise $\eta$ to the input, where $\| \eta \|_\infty $ is "small" in a specific sense. The authors write

In many problems, the precision of an individual input feature is limited. For example, digital images often use only 8 bits per pixel so they discard all information below 1/255 of the dynamic range. Because the precision of the features is limited, it is not rational for the classifier to respond differently to an input $x$ than to an adversarial input $ \tilde{x} = x + \eta$ if every element of the perturbation $\eta$ is smaller than the precision of the features. Formally, for problems with well-separated classes, we expect the classifier to assign the same class to $x$ and $\tilde{x}$ so long as $\|\eta\|_\infty < \epsilon$ , where $\epsilon$ is small enough to be discarded by the sensor or data storage apparatus associated with our problem.

The FGSM method uses the sign because the goal is to create modifications to the input that add up to a misclassification, but are still "small enough." Using the full gradient information causes a larger change to the input, which doesn't satisfy the constraint on $\| \eta \|_\infty$.

Consider the dot product between a weight vector $w$ and an adversarial example $x$
$$
w^T \tilde{x} = w^T x + w^T \eta
$$
The adversarial perturbation causes the activation to grow by $w^T\eta$. We can maximize this increase subject to the max norm constraint on $\eta$ by assigning $\eta = \text{sgn}(w)$. If $w$ has $n$ dimensions and the average magnitude of an element of the weight vector is $m$, then the activation will grow by $\epsilon mn$. Since $\|\eta \|_\infty$ does not grow with the dimensionality of the problem, but the change in activation caused by perturbation by $\eta$ can grow linearly with $n$, then for high dimensional problems, we can make many infinitesimal changes to the input that add up to one large change to the output. We can think of this as a sort of “accidental steganography,” where a linear model is forced to attend exclusively to the signal that aligns most closely with its weights, even if multiple signals are present and other signals have much greater amplitude.

If you're working on a problem that doesn't care about the constraint on $\| \eta \|_\infty$, then it's perfectly reasonable to examine alternative methods. Some good examples appear in "Motivating the Rules of the Game for Adversarial Research" by Goodfellow et al.
