# Rasmussen Equation 5.9

Can any one add the steps showing how Rasmussen (Gaussian Processes for Machine Learning, the MIT Press, 2006) got from line 1 to line 2 of equation 5.9. (pg 114)? It is calculating the gradient of the log-likelihood.

\begin{aligned} \frac{d}{d\theta_j} \log p(y|X, \theta) &= \frac{1}{2}y^T K^{-1}\frac{dK}{d\theta_j}K^{-1}y - \frac{1}{2}\text{tr}(K^{-1}\frac{dK}{d\theta_j})\\ &= \frac{1}{2}\text{tr}\Bigl((\alpha\alpha^T-K^{-1})\frac{dK}{d\theta_j}\Bigl) \end{aligned}

Where: $$\alpha=K^{-1}y$$ and $$K$$ is PSD

It is due to the cyclic property of the trace:

$$\mathrm{tr}ABC = \mathrm{tr}BCA = \mathrm{tr}CAB$$

Substituting $$A = \alpha^T$$, $$B = dK/d\theta_j$$, and $$C = \alpha$$, and performing the rotation above (from $$ABC$$ to $$CAB$$) gives us:

$$\alpha^T{dK\over d\theta_j}\alpha = \mathrm{tr}\left(\alpha^T{dK\over d\theta_j}\alpha\right) = \mathrm{tr}\left(\alpha \alpha^T{dK \over d\theta_j}\right)$$

where the first equality is due to the fact that $$\alpha^T{dK\over d\theta_j}\alpha$$ is a scalar, hence equal to its own trace.

We can get the rest of the way there by using the additive properties of the trace:

$$\mathrm{tr}A + \mathrm{tr}B =\mathrm{tr}(A+B)$$