5
$\begingroup$

Can any one add the steps showing how Rasmussen (Gaussian Processes for Machine Learning, the MIT Press, 2006) got from line 1 to line 2 of equation 5.9. (pg 114)? It is calculating the gradient of the log-likelihood.

\begin{equation} \begin{aligned} \frac{d}{d\theta_j} \log p(y|X, \theta) &= \frac{1}{2}y^T K^{-1}\frac{dK}{d\theta_j}K^{-1}y - \frac{1}{2}\text{tr}(K^{-1}\frac{dK}{d\theta_j})\\ &= \frac{1}{2}\text{tr}\Bigl((\alpha\alpha^T-K^{-1})\frac{dK}{d\theta_j}\Bigl) \end{aligned} \end{equation}

Where: $\alpha=K^{-1}y$ and $K$ is PSD

$\endgroup$

1 Answer 1

7
$\begingroup$

It is due to the cyclic property of the trace:

$$\mathrm{tr}ABC = \mathrm{tr}BCA = \mathrm{tr}CAB$$

Substituting $A = \alpha^T$, $B = dK/d\theta_j$, and $C = \alpha$, and performing the rotation above (from $ABC$ to $CAB$) gives us:

$$\alpha^T{dK\over d\theta_j}\alpha = \mathrm{tr}\left(\alpha^T{dK\over d\theta_j}\alpha\right) = \mathrm{tr}\left(\alpha \alpha^T{dK \over d\theta_j}\right)$$

where the first equality is due to the fact that $\alpha^T{dK\over d\theta_j}\alpha$ is a scalar, hence equal to its own trace.

We can get the rest of the way there by using the additive properties of the trace:

$$\mathrm{tr}A + \mathrm{tr}B =\mathrm{tr}(A+B)$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.