# Distribution of the Sum of an AR(1) Model Time Series

I have the following model for my model

$$\Delta X_{t} = \mu \Delta t + \rho \Delta X_{t-1} + \sigma \sqrt{\Delta t} Z_t$$

with the following initial conditions -

$$\Delta X_{1} = \mu \Delta t + \sigma \sqrt{\Delta t} Z_1$$

and

$$X_{0} = Z_0$$

all the $$Z$$ variables are i.i.d Standard normal variables $$\sim \mathcal{N}(0,1)$$

My goal is to find the distribution of $$X_T$$

This is my progress so far -

$$\begin{bmatrix} X_{T} \\ X_{T-1} \\ X_{T-2} \\ \vdots \\ X_2 \\ X_1 \\ X_0 \end{bmatrix} = \mu \Delta t \begin{bmatrix} 1 \\ 1 \\ 1 \\ \vdots \\ 1 \\ 1 \\ 0 \end{bmatrix} + \begin{bmatrix} 0 & 1+\rho & -\rho & \cdots & 0 & 0 & 0 \\ 0 & 0 & 1+\rho & \cdots & 0 & 0 & 0 \\ 0 & 0 & 0 & \cdots & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & \cdots & 0 & 1+\rho & -\rho \\ 0 & 0 & 0 & \cdots & 0 & 0 & 1 \\ 0 & 0 & 0 & \cdots & 0 & 0 & 0 \end{bmatrix} \begin{bmatrix} X_{T} \\ X_{T-1} \\ X_{T-2} \\ \vdots \\ X_2 \\ X_1 \\ X_0 \end{bmatrix} + \begin{bmatrix} \sigma \sqrt{\Delta t} & 0 & 0 & \cdots & 0 & 0 & 0 \\ 0 & \sigma \sqrt{\Delta t} & 0 & \cdots & 0 & 0 & 0 \\ 0 & 0 & \sigma \sqrt{\Delta t} & \cdots & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & \cdots & \sigma \sqrt{\Delta t} & 0 & 0 \\ 0 & 0 & 0 & \cdots & 0 & \sigma \sqrt{\Delta t} & 0 \\ 0 & 0 & 0 & \cdots & 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} Z_{T} \\ Z_{T-1} \\ Z_{T-2} \\ \vdots \\ Z_2 \\ Z_1 \\ Z_0 \end{bmatrix}$$

Condensing it I get

$$\mathbf{x} = (\mu \Delta t) \mathbf{k} + \mathbf{R} \mathbf{x} + \mathbf{S} \mathbf{z}$$

Simplifying the expression, I get -

$$\mathbf{x} = \left(\mathbf{I} - \mathbf{R} \right)^{-1}\left( (\mu \Delta t) \mathbf{k} + \mathbf{S} \mathbf{z}\right)$$

This works but is very inefficient to code up with the matrix inversion. Any way I can get the closed form solution for $$X_T$$? Or at least a more efficient way to compute it?

EDIT 1: I managed to get the expected value using the following steps.

$$X_T = X_0 + \Delta X_1 + \Delta X_2 + \cdots + \Delta X_T$$

If we look at the expected value of any of the $$\Delta X_t$$ variables we see

$$\mathbb{E}\left[\Delta X_t \right] = \mu \Delta t + \rho (\mathbb{E}\left[\Delta X_{t-1} \right] )$$

This, in combination with the initial condition, gives me -

$$\mathbb{E}\left[\Delta X_1 \right] = \mu \Delta t$$

$$\mathbb{E}\left[\Delta X_2 \right] = \mu \Delta t + \rho (\mu \Delta t)$$

$$\mathbb{E}\left[\Delta X_T \right] = \mu \Delta t + \rho (\mu \Delta t) + \cdots + \rho^{T-1}(\mu \Delta t)$$

Adding them all up, I get -

$$\sum_{i=1}^{T}\mathbb{E}\left[\Delta X_1 \right] = \sum_{i=1}^{T} \mu \rho^{i-1} (T-i+1) \Delta t$$

After multiplying this equation with $$\rho$$, subtracting from itself, and simplifying the equation, I get

$$\sum_{i=1}^{T}\mathbb{E}\left[\Delta X_1 \right] = \left(\frac{\mu \Delta t}{1- \rho} \right)\left[T - \frac{1-\rho^{T+1}}{1- \rho} \right]$$

Finally, I have the expected value of $$X_T$$ as

$$\mathbb{E}\left[\Delta X_T \right] = \mathbb{E}\left[\Delta X_0 \right] + \sum_{i=1}^{T}\mathbb{E}\left[\Delta X_1 \right] = \sum_{i=1}^{T}\mathbb{E}\left[\Delta X_1 \right]$$

Giving me

$$\mathbb{E}\left[\Delta X_T \right] \sim \mathcal{N}\left( \left(\frac{\mu \Delta t}{1- \rho} \right)\left[T - \frac{1-\rho^{T+1}}{1- \rho} \right] , ?? \right)$$

Still trying to figure out the variance.

EDIT 2: I followed the same process for the variance.

$$Var\left( \Delta X_t \right) = \rho^{2} Var( \Delta X_{t-1} ) + \sigma^2 \Delta t$$

Like with the expected values, this gives me.

$$Var\left( \Delta X_1 \right) = \sigma^2 \Delta t$$

$$Var\left( \Delta X_2 \right) = \sigma^2 \Delta t + \rho^{2} (\sigma^2 \Delta t)$$

$$Var\left( \Delta X_T \right) = \sigma^2 \Delta t + \rho^{2} (\sigma^2 \Delta t) + \cdots \rho^{2T-2} (\sigma^2 \Delta t)$$

We also have to take into account the covariances that exist

$$Cov(\Delta X_t , X_{t-1}) = \rho Var(\Delta X_{t-1})$$

This gives us

$$Cov(\Delta X_2 , X_1) = \rho (\sigma^2 \Delta t)$$

$$Cov(\Delta X_3 , X_2) = \rho (\sigma^2 \Delta t) + \rho^3 (\sigma^2 \Delta t)$$

$$Cov(\Delta X_T , X_{T-1}) = \rho (\sigma^2 \Delta t) + \rho^3 (\sigma^2 \Delta t) + \cdots + \rho^{2T-3} (\sigma^2 \Delta t)$$

Putting it all together we get

$$Var\left( \sum_{i=1}^{T} \Delta X_1 \right) = \sigma^2 \Delta t \left( T + \sum_{i=1}^{T-1} (T-i)( 2 \rho^{2i-1} + \rho^{2i} ) \right)$$

Solving for $$\sum_{i=1}^{T-1} (T-i)( 2 \rho^{2i-1} + \rho^{2i} )$$ we get

$$Var\left( \sum_{i=1}^{T} \Delta X_1 \right) = \sigma^2 \Delta t \left( T + \frac{1}{1-\rho^2}\left( (T-1)(2\rho^{1} + \rho^{2}) - \frac{1-\rho^{2T-2}}{1-\rho^2}(2\rho^3 + \rho^4) \right) \right)$$

Since $$X_0$$ is independent we get the variance of $$X_T$$ as

$$Var(X_T) = 1 + \sigma^2 \Delta t \left( T + \frac{1}{1-\rho^2}\left( (T-1)(2\rho^{1} + \rho^{2}) - \frac{1-\rho^{2T-2}}{1-\rho^2}(2\rho^3 + \rho^4) \right) \right)$$

Final distribution of $$X_T$$ is

$$\mathcal{N}\left( \left(\frac{\mu \Delta t}{1- \rho} \right)\left[T - \frac{1-\rho^{T+1}}{1- \rho} \right] , 1 + \sigma^2 \Delta t \left( T + \frac{1}{1-\rho^2}\left( (T-1)(2\rho^{1} + \rho^{2}) - \frac{1-\rho^{2T-2}}{1-\rho^2}(2\rho^3 + \rho^4) \right) \right)\right)$$

• Because $R$ is nilpotent ($R^{T+1}=0$), inverting it is easy to do via $$(I-R)^{-1}=I+R+R^2+\cdots+R^T.$$ However, it's usually inefficient to invert matrices directly: simply solve the equation $$(I-R)\mathbf{x} = (\mu\Delta t)\mathbf{k} + \mathbf{Sz}.$$ You don't need to go this far, though, because all you really need are the mean and covariance of $x_T$--you know the distribution is Normal. – whuber May 24 at 20:42