# Sufficient statistic for Gaussian $AR(1)$

Question Does the Gaussian $$AR(1)$$ model, with a fixed sample size $$T$$, have nontrivial sufficient statistics?

The model is given by $$y_t = \rho y_{t-1}, \, t = 1, \cdots, T, \; \epsilon_i \stackrel{i.i.d.}{\sim} \mathcal{N}(0, \sigma^2),$$ parametrized by $$(\rho, \sigma^2)$$. Let $$y = (y_1, \cdots, y_T)'$$. Conditional on $$y_0$$, the likelihood function is \begin{align*} L(y) &= \frac{1}{ ( \sqrt{2 \pi \sigma^2} )^n } e^{-\frac{1}{2\sigma^2} ( \sum_{t = 1}^T (y_t - \rho y_{t-1})^2)} \\\\ &= \frac{1}{ ( \sqrt{2 \pi \sigma^2} )^n } e^{-\frac{1}{2\sigma^2} y' P(\rho) y}, \end{align*} where $$P(\rho)$$ is the $$T \times T$$ tridiagonal Toeplitz matrix given by $$P(\rho) = \begin{bmatrix} 1 + \rho^2 & -\rho & 0 & 0 & & & \\ -\rho & 1 + \rho^2 & -\rho & 0 & & & \\ 0 &-\rho & 1 + \rho^2 & -\rho & & & \\ & & & & \ddots& & \\ & & & & -\rho & 1 + \rho^2 & -\rho\\ & & & & 0 & -\rho & 1\\ \end{bmatrix}.$$

My guess is that, if a nontrivial sufficient statistic exists, it would involve factorizing $$P(\rho)$$.

Comment

This situation is a little different with the case of the Gaussian linear model $$Y = X \beta + \epsilon, \;\; \epsilon \stackrel{d}{\sim} \mathcal{N}(0, \sigma^2 I_T)$$ parametrized by $$\beta \in \mathbb{R}^p$$ and $$\sigma^2 > 0$$. The design matrix $$X, T \times p$$ is considered fixed. The likelihood function is \begin{align*} L(y) &= \frac{1}{ ( \sqrt{2 \pi \sigma^2} )^n } e^{-\frac{1}{2\sigma^2} ( Y - X \beta)'( Y - X \beta) } \\\\ &= \frac{1}{ ( \sqrt{2 \pi \sigma^2} )^n } e^{-\frac{1}{2\sigma^2} \left[ ( Y - X \hat{\beta})'( Y - X \hat{\beta}) + (\hat{\beta} - \beta)' X' X (\hat{\beta} - \beta) \right]}. \end{align*} This makes $$(\hat{\beta}, s^2)$$ sufficient (and minimal), where $$\hat{\beta}$$ is the OLS estimate $$\hat{\beta}$$ and $$s^2 = \frac{1}{T-1} ( Y - X \hat{\beta})'( Y - X \hat{\beta})$$. But in the $$AR(1)$$ case, it doesn't make sense to consider the covariates $$y_{t-1}$$ as fixed.

Consider the conditional form of the likelihood function is (still conditioning on $$y_0$$) \begin{align*} L(y) &= \frac{1}{ ( \sqrt{2 \pi \sigma^2} )^n } e^{-\frac{1}{2\sigma^2} ( \sum_{t = 1}^T (y_t - \rho y_{t-1})^2)}. \\\\ \end{align*}
As in the case of linear model $$\sum_{t = 1}^T (y_t - \rho y_{t-1})^2 = \sum_{t = 1}^T (y_t - \hat{\rho} y_{t-1})^2 + (\hat{\rho} - \rho)^2 \sum_{t = 1}^T y_{t-1}^2$$ where $$\hat{\rho} = \frac{\sum_{t = 1}^T y_{t-1} y_t}{\sum_{t = 1}^T y_{t-1}^2}$$ is the OLS estimate, and $$\sum_{t = 1}^T (y_t - \hat{\rho} y_{t-1})^2$$ is the residual sum of squares.
Evidently, $$T_1 = (\sum_{t = 1}^T (y_t - \hat{\rho} y_{t-1})^2,\, \hat{\rho},\, \sum_{t = 1}^T y_{t-1}^2)$$ is a minimal sufficient statistic.
The only difference with the linear model case is the additional term $$\sum_{t = 1}^T y_{t-1}^2 = X'X$$. This is not surprising. Inspecting the calculation for the linear model, we see that taking the design matrix $$X$$ as fixed is unnecessary. One only needs to take $$X'X$$ as fixed. In the time series case, the statement "taking $$\sum_{t = 1}^T y_{t-1}^2$$ as fixed" doesn't make sense. Instead it appears as part of a sufficient statistic.
Equivalently, \begin{align*} \sum_{t = 1}^T (y_t - \rho y_{t-1})^2 &= \sum_{t = 1}^T y_t^2 - 2 \rho \sum_{t = 1}^T y_{t-1} y_t + \rho^2 \sum_{t = 1}^T y_{t-1}^2 \\\\ &= (1 + \rho^2) \sum_{t = 1}^T y_{t-1}^2 + y_T^2 - 2 \rho \sum_{t = 1}^T y_{t-1} y_t \end{align*} means $$T_2 = (\sum_{t = 1}^T y_{t-1}^2,\, y_T^2,\, \sum_{t = 1}^T y_{t-1} y_t)$$ is another minimal sufficient statistic. Geometrically, it is clear that $$T_2$$ contains the same information as $$T_1 = \mbox{ (sum of OLS square residuals, OLS estimate, squared norm of (y_{t-1})). }$$