I was able to use the following equation to find derivatives of matrix function:

$f(X+h) = f(X) + Ah + o(|h|) \quad \cdots (1)$

where $h$ is small displacement and $A$ is the jacobian matrix. I found couple more equations that find derivatives of matrix function:

$D_Yf(X) = \lim_{t->0} \frac{f(X+tY) - f(X)}{t} \quad \cdots (2)$

$D_Yf(X) = \lim_{t->0} \frac{f(X+tY) - f(X)}{t} = tr(Y^TU) \quad \cdots (3)$

It seems like the first equation and second equation are identical with (2) being more precise in terms of the definition of a derivative( Equation (2) also shows why $|h|^2$ is ignored). Furthermore, both (1) and (2) apply to functions $\mathbb{R}^n$->$\mathbb{R}^m$. Equation (3) seems to be special case of (2). Equation (3) was used like the following:

$f(X) = tr(AX)$ $D_Yf(X) = \lim_{t->0} \frac{f(X+tY)-f(X)}{t} \\ = \lim_{t->0} \frac{tr(A(X+tY) - tr(AX)}{t} \\ = \lim_{t->0} \frac{tr(AX+AtY] - tr(AX)}{t} \\ = \lim_{t->0} \frac{tr(tAY)}{t} \\ = \lim_{t->0} tr(AY)\\ = tr(AY) \\ = tr([AY]^T) \\ = tr(Y^TA^T)$

$U=A^T$, therefore $D_Yf(X) = A^T$.

Couple questions regarding those three equations:

  1. Is $h$ in eq. (1) same as $tY$ in equation (2)?(something seems a bit missing in equation(1) to me...)

  2. I don't quite understand what $tr(Y^TU)$ meansin equation (3). $Y$ seems to be a directional matrix and $U$ seems to be the jacobian but what exactly does that expression mean? And how does formatting into $tr(Y^TU)$ form give us the derivative?

EDIT: they(where i found equation (3)) used column vector for the gradient. (http://www.tc.umn.edu/~nydic001/docs/unpubs/Schonemann_Trace_Derivatives_Presentation.pdf)


closed as unclear what you're asking by whuber Oct 7 '17 at 16:48

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ You should read the matrix cookbook. It covers all this stuff. www2.imm.dtu.dk/pubdb/views/edoc_download.php/3274/pdf/… $\endgroup$ – gammer Jan 24 '17 at 5:46
  • 1
    $\begingroup$ @gammer matrix cookbook seems, to me at least, like a cheat sheet with all the derivations without proof. I want to know how the cookbook derived certain stuff. In fact I was working on algebraically proving equations 33-45 in the cookbook and got stuck with this notion of deriving a trace. $\endgroup$ – MoneyBall Jan 24 '17 at 5:52
  • $\begingroup$ Cool, well check it out if you want. Good luck. $\endgroup$ – gammer Jan 24 '17 at 6:20
  • $\begingroup$ Because I addressed (2) in detail, starting from the definitions, in your previous question at stats.stackexchange.com/questions/257579/…, and because $tY$ in $(2)$ obviously plays exactly the same role as $h$ in $1$, I don't understand what else you need to know. $\endgroup$ – whuber Jan 24 '17 at 16:17
  • $\begingroup$ @whuber okay so you're saying $tY$ is the same as $h$ so that confirms my first question. what exactly is $tr(Y^TU)$? Is U a jacobian to directional matrix Y? Then what about the trace? $\endgroup$ – MoneyBall Jan 24 '17 at 23:12