Suppose that $\theta(t)\sim\mathcal N(\mu(t),\Sigma(t))$ where $t$ is some parameter. Then it holds that $$\theta(t) = \mu(t) + \Sigma(t)^{0.5}\xi$$ for $\xi\sim\mathcal N(0, I)$. I am interested in the mean and variance of $\frac{\mathrm d}{\mathrm dt}\theta(t)$. Assuming $\frac{\mathrm d}{\mathrm dt}\theta(t)\sim\mathcal N(\lambda(t),\Omega(t))$ does it follow that $$\frac{\mathrm d}{\mathrm dt}\theta(t) = \underbrace{\frac{\mathrm d}{\mathrm dt}\mu(t)}_{=\lambda(t)} + \underbrace{\frac{\mathrm d}{\mathrm dt}\Sigma(t)^{0.5}}_{=\Omega(t)^{0.5}}\xi?$$

  • $\begingroup$ As far as I see from gaussianprocess.org/gpml/chapters/RW9.pdf, it's going to be the second derivative of $\Sigma(t_1,t_2)$ wrt $t_1,t_2$ $\endgroup$
    – gunes
    Mar 29, 2020 at 16:47
  • $\begingroup$ So if I understand you correctly, $\Omega(t)^{0.5} = \frac{\mathrm d^2}{\mathrm dt^2}\Sigma(t)^{0.5}$ ? But how can I conclude that the second equation above is wrong then? $\endgroup$ Mar 29, 2020 at 18:28

1 Answer 1


I'll provide the derivation for a general random process (with finite second moments). As the author's random process is a simple random process (stochastic part doesn't depend on time), the direct differentiation (what the author did) is also correct.

The derivation is based on the book [Natan, Gorbachev, Guz, The basics of random process theory, 2003] (in Russian). Also, the result follows from the Rasmussen book, formula (9.1).

Our goal is to find the correlation function $R_{\theta'(t)}(t_1, t_2)$ for $\theta'(t) = \frac{\partial}{\partial t} \theta(t)$: $$ R_{\theta'}(t_1, t_2) = \mathbb{E} (\theta'(t_1) - \mathbb{E} \theta'(t_1)) (\theta'(t_2) - \mathbb{E} \theta'(t_2)) $$

Assume without loss of generality $\mu(t) = 0$. Then for the correlation function of the derivative, we have:

\begin{align*} &R_{\theta'}(t_1, t_2) = \mathbb{E} \theta'(t_1) \theta'(t_2) = \\ & \mathbb{E} \lim_{k \rightarrow 0} \frac{\theta(t_1 + k) - \theta(t_1)}{k} \lim_{l \rightarrow 0} \frac{\theta(t_2 + l) - \theta(t_2)}{l} = \\ & \lim_{k, l \rightarrow 0} \mathbb{E} \frac{(\theta(t_1 + k) - \theta(t_1))(\theta(t_2 + l) - \theta(t_2))}{kl} = \\ & \frac{\partial^2 R_{\theta'}(t_1, t_2) }{\partial t_1 \partial t_2}. \end{align*} Here we use as $\lim$ the mean-square and use additional technical Lemma to move the expectation inside the limit.

To get the variance we set $t_1 = t_2$: $$ \Omega(t) = \left. \frac{\partial^2 R_{\theta}(t_1, t_2) }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t}. $$ For our problem: $$ R_{\theta}(t_1, t_2) = \mathbb{E} \theta(t_1) \theta(t_2) = \sqrt{\Sigma(t_1) \Sigma(t_2)}. $$

So, \begin{align*} \Omega(t) &= \left. \frac{\partial^2 R_{\theta}(t_1, t_2) }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t} = \left. \frac{\partial^2 \sqrt{\Sigma(t_1) \Sigma(t_2)} }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t} = \\ &= \left. \frac14 \frac{1}{\sqrt{\Sigma(t_1) \Sigma(t_2)}} \Sigma'(t_1) \Sigma'(t_2) \right|_{t_1 = t_2 = t} = \\ &= \frac14 \frac{(\Sigma'(t))^2}{\Sigma(t)}. \end{align*}

Using the designation $\sigma(t) = \sqrt{\Sigma(t)}$, we get a simpler expression: $$ \Omega(t) = (\sigma'(t))^2. $$

  • $\begingroup$ wow, thanks for the dervation it makes it much more clear now. However, one additional remark: in my case $\Sigma(t)$ is a n×n matrix. I suppose in your derivation you assumed a scalar variance? $\endgroup$ Mar 30, 2020 at 13:23
  • $\begingroup$ @SydAmerikaner I assumed scalar variable, but the derivation should be the same for all elements of the correlation matrix $R(t_1, t_2)$ $\endgroup$ Mar 30, 2020 at 20:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.