3
$\begingroup$

My question comes from a comment in this question Vector Jacobian product in automatic differentiation

The question states...

$$ t = Wz, \,\,\, z\in \mathbb{R}^{m\times 1}, t \in \mathbb{R}^{n \times 1}, W\in\mathbb{R}^{n \times m} $$

$$ \frac{\partial t}{\partial z} = W $$

Which is all good but then a comment states an observation that a different Jacobian, $\frac{\partial t}{\partial W} \in \mathbb{R}^{n \times n \times m}$. I cannot justify to myself wwhy that Jacobian would have three axes. Can anyone explain this?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Well, $t$ is a vector, and $W$ is a matrix. Think about differentiating everything element-wise, i.e. $$D_{ijk}=\frac{\partial t_i}{\partial W_{jk}}$$ and $D$ automatically becomes a 3D tensor.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Oh yes that makes perfect sense now, IDK why I couldn't see that. Thanks $\endgroup$
    – Joff
    Commented Jun 25, 2021 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.