# Fast Gradient Sign Method First Order Derivative

Given the infinity norm for image $$x$$ and its perturbed image $$x\prime$$ as in $$R_{\infty}\left( x',x \right) =\lVert x-x' \rVert _{\infty}$$, the adverserial attack:

$$attack: x^* \gets arg\max _{x^*:R\left( x,x' \right) \le \epsilon}\mathscr{L}\left(x\prime,y \right)$$

Then we will make a first-order assumption for the loss of the perturbed image (which is first-order Taylor expansion) that locally our loss function:

$$\mathscr{L}\left( x',y \right) \approx \mathscr{L}\left( x,y \right) +\left( x'-x \right) ^T\nabla _x\mathscr{L}$$

To carry out the first-order attack, all we do is substitute $$\left(x' - x \right)^T\nabla _x\mathscr{L}$$ with the loss function in the original equation of the attack. The equation below says that we want every element of $$x$$ to go in the direction of the gradient:

$$attack:x^* \gets arg\max_{x^*:R\left(x,x' \right) \le \epsilon} \left( x'-x \right) ^T\nabla _x\mathscr{L}$$

Question: why we did not substitute $$\mathscr{L}\left( x,y \right) +\left( x'-x \right) ^T\nabla _x\mathscr{L}$$ instead of $$\left( x'-x \right) ^T\nabla _x\mathscr{L}$$ please?

The value $$\mathscr L(x,y)$$ is constant with respect to $$x^*$$ or $$x^\prime$$, so optimizing $$x^*$$ or $$x^\prime$$ won't change $$\mathscr L(x,y)$$.
• Thank you. So you mean the the loss $\mathscr{L}(x,y)$ will be constant when we take the derivative of the loss with respect to the perturbed image $x\prime$ please?