1
$\begingroup$

I have this equation:

$$\nabla f^T x+ \nabla f^T \Sigma^{-1} (\Sigma \circ Q)x = -\frac{1}{2}\nabla f^T \Sigma^{-1} \nabla \tag{1}f$$

where $\nabla f,x$ are vectors, and

$$\nabla f_i =a_i - E[\nabla r_i]$$

($a_i$ being a scalar)

And $\Sigma$ is the covariance matrix corresponding to the vector $\nabla r$, the "$\circ$" indicates the hadamard product, and the elements of $Q$ are covariance elasticities w.r.t. $x$ defined as:

$$Q_{ij} = \frac{1}{\delta_{ij} + 1}\frac{x_i}{\sigma_{ij}}\frac{\partial \sigma_{ij}}{\partial x_i}$$

(where the factor involving the kronecker delta is there just to get rid of a 2 which appears in the diagonal terms.)

So, it must be kept in mind that each $i,j$ element of $\Sigma$ (and hence of $\Sigma^{-1}$ and $Q$) is a function of $x_i$ and $x_j$.

I want to differentiate equation 1 w.r.t. $\Sigma^{-1}$.

I start with the "trace trick".

$$\nabla f^T x + tr((\nabla f x^T) \Sigma^{-1} (\Sigma \circ E)) = -\frac{1}{2}tr(\nabla f \nabla f^T \Sigma^{-1})$$

Then, I invoke the trace's invariance under cyclic permutations to rewrite the second term on the LHS as follows:

$$\nabla f^T x + tr((\Sigma \circ E) (\nabla f x^T) \Sigma^{-1}) = -\frac{1}{2}tr(\nabla f \nabla f^T \Sigma^{-1})$$

Now everything is set up to apply the basic formula for differentiation of a trace with respect to a matrix (Eq. 100 in the matrix cookbook):

$$(\Sigma \circ E) (\nabla f x^T)=-\frac{1}{2}\nabla f \nabla f^T$$

My main question here is what about $\Sigma \circ Q$? I am basically praying that it does not participate in the differentiation, but that doesn't seem likely since all of these matrices are functions of the same vector $x$.

Secondly, note that, alternatively, you can write equation 1 as:

$$\nabla f^T x+ \nabla f^T \Sigma^{-1} z = -\frac{1}{2}\nabla f^T \Sigma^{-1} \nabla f$$

where $z=(\Sigma \circ Q)x$

Following the same steps as above (and again praying that $\frac{\partial (\Sigma \circ Q)}{\partial \Sigma^{-1}} = 0$) you then arrive at

$$\nabla f z^T=-\frac{1}{2}\nabla f \nabla f^T$$

Which seems to mean that $z=-\frac{1}{2}\nabla f$, and then plugging this back into Eq. 1 gives $\nabla f^Tx =0$?

$\endgroup$
1
$\begingroup$

Let's use a colon to denote the trace/Frobenius product $$A:B={\rm tr}(A^TB)$$ Define the variables $$\eqalign{ g &= \nabla f,\,\,\,\,\,\alpha=g^Tx,\,\,\,\,\,M = M^T = \Sigma^{-1} \cr \phi &= \alpha + (gx^T):(M(Q\circ M^{-1})) + \tfrac{1}{2}(gg^T):M \,\,\,= 0 \cr }$$ You wish to find the gradient $\frac{\partial\phi}{\partial M}$, start by finding its differential. $$\eqalign{ d\phi &= (gx^T):(dM(Q\circ M^{-1})) + (gx^T):(M(Q\circ dM^{-1})) + \tfrac{1}{2}(gg^T):dM \cr &= (gx^T(Q\circ M^{-1})^T):dM + (Q\circ(Mgx^T)):dM^{-1} + \tfrac{1}{2}(gg^T):dM \cr &= \Big(gx^T(Q\circ M^{-1})^T-M^{-1}\big(Q\circ(Mgx^T)\big)M^{-1}+\tfrac{1}{2}gg^T\Big):dM \cr }$$ Setting the gradient to zero $$\eqalign{ \frac{\partial\phi}{\partial M} &= gx^T(Q\circ M^{-1})^T-M^{-1}\big(Q\circ(Mgx^T)\big)M^{-1}+\tfrac{1}{2}gg^T = 0 }$$ leaves us with $$\eqalign{ -\tfrac{1}{2}gg^T &= gx^T(Q\circ M^{-1})^T-M^{-1}\big(Q\circ(Mgx^T)\big)M^{-1} \cr -\tfrac{1}{2}\nabla f\,\nabla f^T &= \nabla fz^T - \Sigma\Big(Q\circ\big(\Sigma^{-1}\nabla f\,x^T\big)\Big)\Sigma \cr }$$ So you derived the first two terms correctly, but missed the 3rd.

$\endgroup$
  • $\begingroup$ In order to remove the ambiguity, could you tell us the intended precedence of "$:$"? That would help readers understand what you're doing without having to guess what you mean. $\endgroup$ – whuber Sep 26 '18 at 22:33
  • $\begingroup$ @whuber I've added explicit parentheses around each term involved in a trace product. Hope that resolves any ambiguities. $\endgroup$ – greg Sep 26 '18 at 23:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.