I am trying to compute the gradient with respect to a vector $\mathrm x \in \mathbb{R}^d$ of a complicated expression involving the trace of matrix product.

The expression is the following: $$ F(\mathrm x) = \text{Tr}\left[ C \nabla (C^{-1} \mathrm f(\mathrm x)) \right],$$ where $C$ is a $d \times d$ matrix, and $\mathrm f(\mathrm x): \mathbb{R}^d \rightarrow \mathbb{R}^d$ a vector valued function.

What I have until now is $$ \nabla F(\mathrm x) = \nabla \text{Tr}\left[ C (C^{-1} J_f(\mathrm x)) \right],$$

where with $J_f(\mathrm x)$ I denote the Jacobian of $\mathrm f$. However I do not know how to proceed and take the gradient of the trace. Would it be like this? $$\nabla \text{Tr}\left[ C (C^{-1} J_f(\mathrm x)) \right] = \text{Tr}\left[\nabla\left( C (C^{-1} J_f(\mathrm x)) \right)\right]$$


2 Answers 2


Unfortunately, your second equation is incorrect.

In index notation, the function $F$ is $$\eqalign{ \def\n{\nabla} \def\p{\partial} \def\T{\operatorname{Tr}} F &= C_{ij}\;\p_j(C_{ik}^{-1} f_k) \\ &= C_{ij} C_{ik}^{-1}\;(\p_j f_k) \\ &= C_{ij} C_{ik}^{-1}\:J_{jk} \\ &= C_{ki}^{-T}C_{ij}\:J_{jk} \\ }$$ which can be expressed in terms of the trace in various ways $$\eqalign{ F &= \T(C^{-T}C\:J) \\ &= \T(J^TC^TC^{-1}) \\ }$$ But the gradient $\n F$ is $$\eqalign{ \p_\ell F &= C_{ij} C_{ik}^{-1}\;(\partial_\ell J_{jk}) }$$ which doesn't have a simple trace expression.

The biggest obstacle is that the gradient is a vector while the trace is a scalar.

$\sf NB\!:\:$ If $\,C=C^T$ then you can proceed as indicated in gunes answer


$CC^{-1}$ cancels each other in your second equation, you'll be left with $$\nabla \text{Tr}[J_f(x)]=\nabla\left(\sum_{i=1}^d \frac{\partial f_i(x)}{\partial x_i}\right)=\sum_{i=1}^d \frac{\partial}{\partial x_i}\nabla f_i(x)$$

Or $\nabla(\nabla\cdot f)$ in vector calculus.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.