3
$\begingroup$

Background

Consider the following AR($p$) model:

$$ \dot{X_t} = \phi_1 \dot X_{t-1} + \phi_2 \dot X_{t-2} + \cdots + \phi_p \dot X_{t-p} + \epsilon $$

where $\dot{X} := X - \mu = X - \mathbb{E}(X)$, and $\epsilon \overset{iid}{\sim} N(0, \sigma_\epsilon^2) $.


Problem

Derive that the asymptotic distribution of $\hat{\phi}:=[\hat{\phi}_1, \cdots, \hat{\phi}_p]$ follows,

$$ \sqrt{n}(\hat{\phi} - \phi) \sim N(0, \sigma_\epsilon^2 \Gamma^{-1}) $$

where $\Gamma := \begin{bmatrix} \gamma_0 & \gamma_1 & \gamma_2 & \cdots & \gamma_{k-1} \\ \gamma_1 & \gamma_0 & \gamma_1 & \cdots & \gamma_{k-2} \\ \gamma_2 & \gamma_1 & \gamma_0 & \cdots & \gamma_{k-3} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \gamma_{k-1} & \gamma_{k-2} & \gamma_{k-3} & \cdots & \gamma_0\\ \end{bmatrix}$

with $\gamma_k := \text{Cov}(X_t, X_{t-k})$ : the autocovariance of lag $k$.


Try

To estimate $\phi_1, \cdots, \phi_p$, we usually consider the Yule-Walker equation,

$$ \mathbf{\rho} := \begin{bmatrix} \rho_1 \\ \rho_2 \\ \vdots \\ \rho_p \end{bmatrix} = \begin{bmatrix} 1 & \rho_1 & \rho_2 & \cdots & \rho_{k-1} \\ \rho_1 & 1 & \rho_1 & \cdots & \rho_{k-2} \\ \rho_2 & \rho_1 & 1 & \cdots & \rho_{k-3} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \rho_{k-1} & \rho_{k-2} & \rho_{k-3} & \cdots & 1 \\ \end{bmatrix} \begin{bmatrix} \phi_1 \\ \phi_2 \\ \vdots \\ \phi_p \end{bmatrix} = \mathbf{P}\mathbf{\phi} $$

where $\phi_i$ : the coefficient of the model, and $\rho_k$ is the autocorrelation function(ACF) of lag $k$.


I think we can use the fact that we can estimate $\mathbf{\phi}$

$$ \hat{\mathbf{\phi}} = \hat{\mathbf{P}}^{-1}\hat{\mathbf{\rho}} $$

where $\hat{\mathbf{\rho}}=[\hat{\rho_1}, \cdots, \hat{\rho_k}]^T$, which is the estimated version of $\rho$.


We can estimate $\rho_k$ which is a function of lag $k$ given data $\mathbf{x}$, as follows.

$$ \hat{\rho}_k := \frac{ \sum_{t=1}^{n-k} (x_t - \bar{x})(x_{t+k} - \bar{x}) }{\sum_{t=1}^n (x_t - \bar{x})^2} $$

where $\bar{x} := \left(\sum_{t=1}^n x_t \right)/n$.

But I cannot proceed to show the result I want,

$$ \sqrt{n}(\hat{\phi} - \phi) \sim N(0, \sigma_\epsilon^2 \Gamma^{-1}) $$

$\endgroup$
4
$\begingroup$

It doesn't have to be thru Yule-Walker. You can use the OLS approach, namely,

$$X_t = \pmb X_{t-1}^T \pmb \phi + n_t , \qquad t = p+1,p+2,\ldots$$ where $\pmb X_{t-1} = [X_{t-1} \ldots X_{t-p} ]$and $\pmb \phi = [\phi_1 \ldots \phi_p] $. Stacking all such equations, we get $$\pmb y = \pmb X \pmb \phi + \pmb n$$ where $\pmb y = [X_{p+1} \ldots X_N]^T$ and $$\pmb X = \begin{bmatrix} \pmb X_{p}^T\\ \pmb X_{p+1}^T\\ \vdots \\ \pmb X_{n-1}^T \end{bmatrix} $$ Using OLS, we have $$\hat{\pmb{\phi}} = (\pmb X^T \pmb X)^{-1} \pmb X^T \pmb y = (\pmb X^T \pmb X)^{-1} \pmb X^T (\pmb X \pmb \phi + \pmb n) = \pmb \phi + (\pmb X^T \pmb X)^{-1} \pmb X^T \pmb n $$ Notice that from the above equation $$\mathsf{E}( \hat{\pmb{\phi}} ) = \pmb \phi +(\pmb X^T \pmb X)^{-1} \underbrace{\mathsf{E} \pmb X^T \pmb n}_0 \pmb \phi$$. and $$\mathsf{var}(\hat{\pmb{\phi}}) = \mathsf{E}( (\hat{\pmb{\phi}} - \pmb\phi)(\hat{\pmb{\phi}} - \pmb\phi)^T ) =(\pmb X^T \pmb X)^{-1} \pmb X^T \mathsf{E} (\pmb n \pmb n^T) \pmb X (\pmb X^T \pmb X)^{-1} $$ assuming white noise with same variance across time $\sigma_\epsilon^2$, we get $$\mathsf{var}(\hat{\pmb{\phi}}) = \mathsf{E}( (\hat{\pmb{\phi}} - \pmb\phi)(\hat{\pmb{\phi}} - \pmb\phi)^T ) =(\pmb X^T \pmb X)^{-1} \pmb X^T \sigma_\epsilon^2 \pmb I \pmb X (\pmb X^T \pmb X)^{-1} = \sigma_\epsilon^2 (\pmb X^T \pmb X)^{-1}(\pmb X^T \pmb X)(\pmb X^T \pmb X)^{-1}=\sigma_\epsilon^2 (\pmb X^T \pmb X)^{-1}$$

Now assuming your AR(p) is strictly stationary and ergodic and that $\mathsf{E}(X_t^4)$ exists, then the central limit theorem applies $$\sqrt{n}( \hat{\pmb{\phi}} - \pmb{\phi}) \longrightarrow N(0,\mathsf{var}(\hat{\pmb{\phi}}))$$

Note that $(\pmb X^T \pmb X)^{-1} = \Gamma^{-1}$

$\endgroup$
  • $\begingroup$ Thank you! But isn't it true that $X^TX \approx N\Gamma$? As far as I can see, for example, $\mathbb{E}[(X^TX)_{11}] =\mathbb{E}[X^2_{p} + X ^2_{p+1} \cdots X^2_{N-1} ] = (N-p)\gamma_0$.. $\endgroup$ – moreblue May 23 at 6:27
  • $\begingroup$ It's just a scaling factor, you're right. Depends if your initial estimator was chosen to be $k. \hat{\alpha}$ or $ \hat{\alpha}$, where $k = \frac{1}{\sqrt{N}}$ something like that $\endgroup$ – Ahmad Bazzi May 23 at 6:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.