# Normal distribution "stable" under derivative?

Suppose that $$\theta(t)\sim\mathcal N(\mu(t),\Sigma(t))$$ where $$t$$ is some parameter. Then it holds that $$\theta(t) = \mu(t) + \Sigma(t)^{0.5}\xi$$ for $$\xi\sim\mathcal N(0, I)$$. I am interested in the mean and variance of $$\frac{\mathrm d}{\mathrm dt}\theta(t)$$. Assuming $$\frac{\mathrm d}{\mathrm dt}\theta(t)\sim\mathcal N(\lambda(t),\Omega(t))$$ does it follow that $$\frac{\mathrm d}{\mathrm dt}\theta(t) = \underbrace{\frac{\mathrm d}{\mathrm dt}\mu(t)}_{=\lambda(t)} + \underbrace{\frac{\mathrm d}{\mathrm dt}\Sigma(t)^{0.5}}_{=\Omega(t)^{0.5}}\xi?$$

• As far as I see from gaussianprocess.org/gpml/chapters/RW9.pdf, it's going to be the second derivative of $\Sigma(t_1,t_2)$ wrt $t_1,t_2$ Mar 29, 2020 at 16:47
• So if I understand you correctly, $\Omega(t)^{0.5} = \frac{\mathrm d^2}{\mathrm dt^2}\Sigma(t)^{0.5}$ ? But how can I conclude that the second equation above is wrong then? Mar 29, 2020 at 18:28

I'll provide the derivation for a general random process (with finite second moments). As the author's random process is a simple random process (stochastic part doesn't depend on time), the direct differentiation (what the author did) is also correct.

The derivation is based on the book [Natan, Gorbachev, Guz, The basics of random process theory, 2003] (in Russian). Also, the result follows from the Rasmussen book, formula (9.1).

Our goal is to find the correlation function $$R_{\theta'(t)}(t_1, t_2)$$ for $$\theta'(t) = \frac{\partial}{\partial t} \theta(t)$$: $$R_{\theta'}(t_1, t_2) = \mathbb{E} (\theta'(t_1) - \mathbb{E} \theta'(t_1)) (\theta'(t_2) - \mathbb{E} \theta'(t_2))$$

Assume without loss of generality $$\mu(t) = 0$$. Then for the correlation function of the derivative, we have:

\begin{align*} &R_{\theta'}(t_1, t_2) = \mathbb{E} \theta'(t_1) \theta'(t_2) = \\ & \mathbb{E} \lim_{k \rightarrow 0} \frac{\theta(t_1 + k) - \theta(t_1)}{k} \lim_{l \rightarrow 0} \frac{\theta(t_2 + l) - \theta(t_2)}{l} = \\ & \lim_{k, l \rightarrow 0} \mathbb{E} \frac{(\theta(t_1 + k) - \theta(t_1))(\theta(t_2 + l) - \theta(t_2))}{kl} = \\ & \frac{\partial^2 R_{\theta'}(t_1, t_2) }{\partial t_1 \partial t_2}. \end{align*} Here we use as $$\lim$$ the mean-square and use additional technical Lemma to move the expectation inside the limit.

To get the variance we set $$t_1 = t_2$$: $$\Omega(t) = \left. \frac{\partial^2 R_{\theta}(t_1, t_2) }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t}.$$ For our problem: $$R_{\theta}(t_1, t_2) = \mathbb{E} \theta(t_1) \theta(t_2) = \sqrt{\Sigma(t_1) \Sigma(t_2)}.$$

So, \begin{align*} \Omega(t) &= \left. \frac{\partial^2 R_{\theta}(t_1, t_2) }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t} = \left. \frac{\partial^2 \sqrt{\Sigma(t_1) \Sigma(t_2)} }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t} = \\ &= \left. \frac14 \frac{1}{\sqrt{\Sigma(t_1) \Sigma(t_2)}} \Sigma'(t_1) \Sigma'(t_2) \right|_{t_1 = t_2 = t} = \\ &= \frac14 \frac{(\Sigma'(t))^2}{\Sigma(t)}. \end{align*}

Using the designation $$\sigma(t) = \sqrt{\Sigma(t)}$$, we get a simpler expression: $$\Omega(t) = (\sigma'(t))^2.$$

• wow, thanks for the dervation it makes it much more clear now. However, one additional remark: in my case $\Sigma(t)$ is a n×n matrix. I suppose in your derivation you assumed a scalar variance? Mar 30, 2020 at 13:23
• @SydAmerikaner I assumed scalar variable, but the derivation should be the same for all elements of the correlation matrix $R(t_1, t_2)$ Mar 30, 2020 at 20:17