6
$\begingroup$

How can I differentiate the following by $\mathbf{W}$ ?

\begin{equation} \mathbf{Y} = (\mathbf{W}^T\mathbf{x} + b)^2 \end{equation}

Where $\mathbf{W} \in \mathcal{R}^{d\times D}$ and $\mathbf(x)\in \mathcal{R}^{d\times 1}$

How to calculate $\partial \mathbf{Y}/\partial \mathbf{W}$ ?

Improve this question
$\endgroup$
  • $\begingroup$ Can you explain what is the context? In this kind of equations you usually differentiate the vector, and the matrix is constant. $\endgroup$ – Aksakal Jan 8 '15 at 15:08
  • $\begingroup$ $\mathbf{W}^T\mathbf{x} + b$ does not make any sense. You cannot add a column vector to a scalar. $\endgroup$ – Rodrigo de Azevedo Oct 14 '16 at 15:56
1
$\begingroup$

Matrix calculus is used in such cases. Your equation looks like it's from OLS (least squares) theory. In those you differentiate by vector $x$ some quadratic forms like $\frac{\partial (x'A'Ax)}{\partial x}$. Look up relevant formulae in my link above.

If you really are up to differentiating by matrices not vectors, you'll end up with tensors. Tensors are fun, but so far I haven't seem them used a lot in statistics. They're ubiquitous in physics, btw. Again, follow the link I gave.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.