Can the non-linear activations of a feed-forward NN with multiple inputs induce otherwise absent dependencies between the gradients of the inputs?

Question

Consider a feed-forward neural net with multiple input vectors $x_1, x_2, ..., x_N$ whose final output $y$ is differentiable with respect to those inputs. Assume the learned representations of each input are combined to produce the final output $y$ by addition.

Suppose we remove the non-linear activations (and any other non-linear function applied between layers) so that the output becomes linear with respect to the inputs. Then the output has the form $y=b+\sum_{i=1}^Nx_iW_i$ for tensors $b$ and $W_i, i\in\{1, ..., N\}$, and $dy/dx_i = W_i^T$ if I remember my matrix calculus.

Can the non-linear activations or functions, and in particular RELU or softmax, change this behavior so that $dy/dx_i$ involves some $W_{j\neq i}$ or $x_{j\neq i}$?

Answer 1

You can make the problem really simple: let $y = f(w_1x_1 + w_2x_2)$ for scalars $w_i, x_i$ and nonlinear function $f$. The derivative with respect to $x_1$ is $w_1f'(w_1x_1 + w_2x_2)$, which depends on everything.