# How to show that a stable discrete stochastic process converges to a stationary process?

So I have a discrete stochastic process defined by

$$x_{k+1}=Ax_k+Bw_k$$

where $$w_k$$ is zero mean Gaussian white noise with covariance $$R_w$$, and where $$A$$ has its eigenvalues in the unit disk.

I can get the mean and covariance updates for $$x$$ as

$$E[x_{k+1}]=AE[x_k]=A^{k+1}E[x_0]$$

$$E[x_{k+1}x_{k+1}^T]=AE[x_{k}x_{k}^T]A^T+BR_w B^T$$

and with $$A$$ having its eigenvalues in the unit disk implying $$A^n$$ goes to $$0$$ as $$n$$ goes to $$\infty$$, I can see how the mean of $$x$$ will converge to $$0$$.

However, apparently the covariance of $$x$$ should also converge making $$x$$ converge to a stationary process, but I am having trouble showing this mainly because the integrated covariance seems to have a summation with an unbounded amount of terms $$A^nBR_wB^T[A^T]^n$$ with different $$n$$.

How can I show that the covariance of $$x$$ will also converge as $$k$$ goes to $$\infty$$?

For $$k \in \mathbb{N}$$ the covariance of $$x_k$$ is given by $$\mathbb{E}\left[ x_k x_k^\top \right] - \mathbb{E}\left[ x_k \right] \mathbb{E}\left[ x_k \right]^\top$$ where

$$\mathbb{E}\left[ x_k x_k^\top \right] = A^{k} \mathbb{E}\left[ x_0 x_0^\top \right] \left( A^k \right)^\top + \sum_{i=0}^k A^i B R_w B^\top \left( A^\top \right)^i$$

and

$$\mathbb{E}\left[ x_k \right] \mathbb{E}\left[ x_k \right]^\top = A^k \mathbb{E}\left[ x_0 \right] \mathbb{E}\left[ x_0 \right]^\top \left( A^\top \right)^k$$

so the covariance is

$$A^{k} \left( \mathbb{E}\left[ x_0 x_0^\top \right] - \mathbb{E}\left[ x_0 \right] \mathbb{E}\left[ x_0 \right]^\top \right) \left( A^k \right)^\top + \sum_{i=0}^k A^i B R_w B^\top \left( A^\top \right)^i$$

As you've correctly identified,

$$k \to \infty \implies A^{k} \left( \mathbb{E}\left[ x_0 x_0^\top \right] - \mathbb{E}\left[ x_0 \right] \mathbb{E}\left[ x_0 \right]^\top \right) \left( A^k \right)^\top \to 0$$

For the remaining terms, we need some functional analysis facts.

• $$\mathbb{R}^{m \times m}$$ with the operator norm is a Banach space.
• The set of PSD matrices $$\mathbb{S}^m_+ \subseteq \mathbb{R}^{m \times m}$$ is also a Banach space.
• In a Banach space $$X$$, every absolutely convergent series converges in $$X$$.

Therefore,

$$\sum_{i=0}^\infty \left \lVert A^i B R_w B^\top \left( A^\top \right)^i \right \rVert < \infty \implies \sum_{i=0}^{\infty} A^i B R_w B^\top \left( A^\top \right)^i \text{ converges in \mathbb{S}_+^m}$$

and it only remains for us to show $$\sum_{i=1}^\infty \left \lVert A^i B R_w B^\top \left( A^\top \right)^i \right \rVert < \infty$$.

$$\sum_{i=0}^\infty \left \lVert A^i B R_w B^\top \left( A^\top \right)^i \right \rVert \leq \left \lVert B R_w B^\top \right \rVert \sum_{i=0}^\infty \lambda_{\max}\left( A \right)^{2i} < \infty$$