# What is the jacobian for a neural network

I know that a Jacobian is a matrix holding all of the first-order derivatives for a vector-valued-function. What is the Jacobian of a neural network, though? What are the inputs and what are the outputs of the function to compute the derivative for?

A Jacobian is quite a general term indeed. Lets take this simple, single-hidden-layer network $$\hat{\boldsymbol{y}} = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$ as an example.
When looking around online, I found most people (e.g. here or here) to refer to the weight updates, i.e. $$\frac{\partial \hat{y}}{\partial W^{(0)}_{ij}}, \frac{\partial \hat{y}}{\partial W^{(1)}_{ij}}, \frac{\partial \hat{y}}{\partial b^{(0)}_i}, \frac{\partial \hat{y}}{\partial b^{(1)}_i}$$
In my opinion, though, the Jacobian of a neural network should rather be the Jacobian of the function that is represented by the neural network, i.e. $$net : \mathbb{R}^m \to \mathbb{R}^n : \boldsymbol{x} \mapsto net(\boldsymbol{x}) = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)}),$$ where $m$ is the dimensionality of the input vectors (the number of features) and $n$ is the dimensionality of the output (the number of classes). The Jacobian of this network would then simply be $\mathbf{J} = \frac{\partial \hat{\boldsymbol{y}}}{\partial{\boldsymbol{x}}}$ with entries $J_{ij} = \frac{\partial \hat{y}_i}{\partial x_j}.$