Normal distribution "stable" under derivative?

Question

Suppose that $\theta(t)\sim\mathcal N(\mu(t),\Sigma(t))$ where $t$ is some parameter. Then it holds that $$\theta(t) = \mu(t) + \Sigma(t)^{0.5}\xi$$ for $\xi\sim\mathcal N(0, I)$. I am interested in the mean and variance of $\frac{\mathrm d}{\mathrm dt}\theta(t)$. Assuming $\frac{\mathrm d}{\mathrm dt}\theta(t)\sim\mathcal N(\lambda(t),\Omega(t))$ does it follow that $$\frac{\mathrm d}{\mathrm dt}\theta(t) = \underbrace{\frac{\mathrm d}{\mathrm dt}\mu(t)}_{=\lambda(t)} + \underbrace{\frac{\mathrm d}{\mathrm dt}\Sigma(t)^{0.5}}_{=\Omega(t)^{0.5}}\xi?$$

Answer 1

I'll provide the derivation for a general random process (with finite second moments). As the author's random process is a simple random process (stochastic part doesn't depend on time), the direct differentiation (what the author did) is also correct.

The derivation is based on the book [Natan, Gorbachev, Guz, The basics of random process theory, 2003] (in Russian). Also, the result follows from the Rasmussen book, formula (9.1).

Our goal is to find the correlation function $R_{\theta'(t)}(t_1, t_2)$ for $\theta'(t) = \frac{\partial}{\partial t} \theta(t)$: $$ R_{\theta'}(t_1, t_2) = \mathbb{E} (\theta'(t_1) - \mathbb{E} \theta'(t_1)) (\theta'(t_2) - \mathbb{E} \theta'(t_2)) $$

Assume without loss of generality $\mu(t) = 0$. Then for the correlation function of the derivative, we have:

\begin{align*} &R_{\theta'}(t_1, t_2) = \mathbb{E} \theta'(t_1) \theta'(t_2) = \\ & \mathbb{E} \lim_{k \rightarrow 0} \frac{\theta(t_1 + k) - \theta(t_1)}{k} \lim_{l \rightarrow 0} \frac{\theta(t_2 + l) - \theta(t_2)}{l} = \\ & \lim_{k, l \rightarrow 0} \mathbb{E} \frac{(\theta(t_1 + k) - \theta(t_1))(\theta(t_2 + l) - \theta(t_2))}{kl} = \\ & \frac{\partial^2 R_{\theta'}(t_1, t_2) }{\partial t_1 \partial t_2}. \end{align*} Here we use as $\lim$ the mean-square and use additional technical Lemma to move the expectation inside the limit.

To get the variance we set $t_1 = t_2$: $$ \Omega(t) = \left. \frac{\partial^2 R_{\theta}(t_1, t_2) }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t}. $$ For our problem: $$ R_{\theta}(t_1, t_2) = \mathbb{E} \theta(t_1) \theta(t_2) = \sqrt{\Sigma(t_1) \Sigma(t_2)}. $$

So, \begin{align*} \Omega(t) &= \left. \frac{\partial^2 R_{\theta}(t_1, t_2) }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t} = \left. \frac{\partial^2 \sqrt{\Sigma(t_1) \Sigma(t_2)} }{\partial t_1 \partial t_2} \right|_{t_1 = t_2 = t} = \\ &= \left. \frac14 \frac{1}{\sqrt{\Sigma(t_1) \Sigma(t_2)}} \Sigma'(t_1) \Sigma'(t_2) \right|_{t_1 = t_2 = t} = \\ &= \frac14 \frac{(\Sigma'(t))^2}{\Sigma(t)}. \end{align*}

Using the designation $\sigma(t) = \sqrt{\Sigma(t)}$, we get a simpler expression: $$ \Omega(t) = (\sigma'(t))^2. $$