Return to Answer

A Jacobian is quite a general term indeed. Lets take this simple, single-hidden-layer network $$\hat{\boldsymbol{y}} = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$ as an example.

When looking around online, I found most people (e.g. here or here) to refer to the weight updates, i.e. $$\frac{\partial \hat{y}}{\partial W^{(0)}_{ij}}, \frac{\partial \hat{y}}{\partial W^{(1)}_{ij}}, \frac{\partial \hat{y}}{\partial b^{(0)}_i}, \frac{\partial \hat{y}}{\partial b^{(1)}_i} $$

In my opinion, though, the Jacobian of a neural network should rather be the Jacobian of the function that is represented by the neural network, i.e. $$net : \mathbb{R}^m \to \mathbb{R}^n : \boldsymbol{x} \mapsto net(\boldsymbol{x}) = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)}),$$ where $m$ is the dimensionality of the input vectors (the number of features) and $n$ is the dimensionality of the output (the number of classes). The Jacobian of this network would then simply be holding $\frac{\partial \hat{\boldsymbol{y}}}{\partial{x_i}}$

A Jacobian is quite a general term indeed. Lets take this simple, single-hidden-layer network $$\hat{\boldsymbol{y}} = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$ as an example.

When looking around online, I found most people (e.g. here or here) to refer to the weight updates, i.e. $$\frac{\partial \hat{y}}{\partial W^{(0)}_{ij}}, \frac{\partial \hat{y}}{\partial W^{(1)}_{ij}}, \frac{\partial \hat{y}}{\partial b^{(0)}_i}, \frac{\partial \hat{y}}{\partial b^{(1)}_i} $$

In my opinion, though, the Jacobian of a neural network should rather be the Jacobian of the function that is represented by the neural network, i.e. $$net : \mathbb{R}^m \to \mathbb{R}^n : \boldsymbol{x} \mapsto net(\boldsymbol{x}) = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)}),$$ where $m$ is the dimensionality of the input vectors (the number of features) and $n$ is the dimensionality of the output (the number of classes). The Jacobian of this network would then simply be $\mathbf{J} = \frac{\partial \hat{\boldsymbol{y}}}{\partial{\boldsymbol{x}}}$ with entries $J_{ij} = \frac{\partial \hat{y}_i}{\partial x_j}.$

A Jacobian is quite a general term indeed. Lets take this simple, single-hidden-layer network $$\hat{\boldsymbol{y}} = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$ as an example

When looking around online, I found most people (e.g. here or here) to refer to the weight updates, i.e. $$\frac{\partial \hat{y}}{\partial W^{(0)}_{ij}}, \frac{\partial \hat{y}}{\partial W^{(1)}_{ij}}, \frac{\partial \hat{y}}{\partial b^{(0)}_i}, \frac{\partial \hat{y}}{\partial b^{(1)}_i} $$

In my opinion, though, the Jacobian of a neural network is rather the Jacobian of the function that is represented by the neural network, i.e. $$net : \mathbb{R}^m \to \mathbb{R}^n : \boldsymbol{x} \mapsto net(\boldsymbol{x}) = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$. The Jacobian of this network would then simply be holding $\frac{\partial \hat{\boldsymbol{y}}}{\partial{x_i}}$

A Jacobian is quite a general term indeed. Lets take this simple, single-hidden-layer network $$\hat{\boldsymbol{y}} = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$ as an example.

When looking around online, I found most people (e.g. here or here) to refer to the weight updates, i.e. $$\frac{\partial \hat{y}}{\partial W^{(0)}_{ij}}, \frac{\partial \hat{y}}{\partial W^{(1)}_{ij}}, \frac{\partial \hat{y}}{\partial b^{(0)}_i}, \frac{\partial \hat{y}}{\partial b^{(1)}_i} $$

In my opinion, though, the Jacobian of a neural network should rather be the Jacobian of the function that is represented by the neural network, i.e. $$net : \mathbb{R}^m \to \mathbb{R}^n : \boldsymbol{x} \mapsto net(\boldsymbol{x}) = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)}),$$ where $m$ is the dimensionality of the input vectors (the number of features) and $n$ is the dimensionality of the output (the number of classes). The Jacobian of this network would then simply be holding $\frac{\partial \hat{\boldsymbol{y}}}{\partial{x_i}}$

A Jacobian is quite a general term indeed. Lets take this simple, single-hidden-layer network $$\hat{\boldsymbol{y}} = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$ as an example

When looking around online, I found most people (e.g. here or here) to refer to the weight updates, i.e. $$\frac{\partial \hat{y}}{\partial W^{(0)}_{ij}}, \frac{\partial \hat{y}}{\partial W^{(1)}_{ij}}, \frac{\partial \hat{y}}{\partial b^{(0)}_i}, \frac{\partial \hat{y}}{\partial b^{(1)}_i} $$

In my opinion, though, the Jacobian of a neural network is rather the Jacobian of the function that is represented by the neural network, i.e. $$net : \mathbb{R}^m \to \mathbb{R}^n : \boldsymbol{x} \mapsto net(\boldsymbol{x}) = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$. The Jacobian of this network would then simply be holding $\frac{\partial \hat{\boldsymbol{y}}}{\partial{x_i}}$

A Jacobian is quite a general term indeed. Lets take this simple, single-hidden-layer network $$\hat{\boldsymbol{y}} = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$ as an example

When looking around online, I found most people (e.g. here or here) to refer to the weight updates, i.e. $$\frac{\partial \hat{y}}{\partial W^{(0)}_{ij}}, \frac{\partial \hat{y}}{\partial W^{(1)}_{ij}}, \frac{\partial \hat{y}}{\partial b^{(0)}_i}, \frac{\partial \hat{y}}{\partial b^{(1)}_i} $$

In my opinion, though, the Jacobian of a neural network is rather the Jacobian of the function that is represented by the neural network, i.e. $$net : \mathbb{R}^m \to \mathbb{R}^n : \boldsymbol{x} \mapsto net(\boldsymbol{x}) = g(\mathbf{W}^{(1)} \cdot f(\mathbf{W}^{(0)} \cdot \boldsymbol{x} + \boldsymbol{b}^{(0)}) + \boldsymbol{b}^{(1)})$$. The Jacobian of this network would then simply be holding $\frac{\partial \hat{\boldsymbol{y}}}{\partial{x_i}}$