# Asymptotic Least Squares question (with random regressors)

Consider the DGP $y_i=x_i+\epsilon_i$, where $\epsilon_i \sim Z$. We estimate $\beta=1$ by regression without a constant term, so in $y_i=\beta x_i + \epsilon_i$.

1. Show that this DGP does not satisfy the stability assumption ($\operatorname{plim}(\frac{1}{n}X'X) \rightarrow Q, Q$ invertible) and show that the speed of convergence of $b$ to $\beta$ is $n\sqrt{n}$ in this case. (Hint: $\sum_{i=1}^n i^2=\frac{1}{6}n(n+1)(2n+1))$
2. Let $x_i=\frac{1}{i}$. Show that again the stability assumption is not satisfied and that the speed of convergence is $n^0$. (Hint: $\sum_{i=1}^\infty \frac{1}{i^2}=\frac{1}{6}\pi^2$).

What I tried for (1): $\frac{1}{n}x'x=\frac{1}{n}\sum x_i^2=\frac{1}{6}(n+1)(2n+1)$. Which does not satisfy the assumption I guess?

Also \begin{align*} n^p(b-\beta) &= (x'x)^{-1} x'\epsilon\\ &= \left(\frac{1}{n^p} x'x\right)^{-1} x'\epsilon\\ &= \frac{6n^p}{n(n+1)(2n+1)} x'\epsilon\end{align*}

And I think that I need to show that if I would take the limit $n\rightarrow \infty$, the first term would evaluate to a constant if $p=\frac{3}{2}$. I don't however see why this is so ($p=3$ would follow from my derivation I believe).

What I tried for (2). $\frac{1}{n}x'x=\frac{1}{n}\sum_{i=1}^n \frac{1}{i^2}$. Again this doesn't satisfy the assumption? And I'm not sure about the second part of this subquestion.

• You forgot to provide the most important part: what are $x_i$'s. They are random variables alright, but what are their characteristics? Do they form a stationary series? What? It all depends on that information. And what does $Z$ stands for? Feb 5, 2014 at 20:09
• Please supply missing details. "DGP" = discrete Gaussian process? Differentiable Gaussian process? Data generating process? Something else? And what does "$\tilde Z$" mean? What is $b$--the ordinarily least squares estimator of $\beta$? And what is the connection between $X$ (a random matrix, evidently) and $x$ (a determined, non-random matrix)?
– whuber
Feb 5, 2014 at 20:12
• First of all, sorry for using a temporary account but I had difficulty logging in to my own account so I was the one to ask the question.
– rbm
Feb 6, 2014 at 7:48
• @alecos That was also what I found confusing, since it is not mentioned in the question. But I think that there are just $n \ x_i$ 's that together form the vector $x$ but I am not sure. Z is the standard normal distribution.
– rbm
Feb 6, 2014 at 7:50
• @whuber DGP is data generating process. Z is the standard normal distribution. b is indeed the OLS as you mentioned. And I let $x$ denote a random vector of the $x_i$'s, but I'm not sure whether my notation is correct.
– rbm
Feb 6, 2014 at 7:54

The original question appears ill-posed, regarding part 1). For arbitrary regressor $X$ the stability condition may or may not be satisfied. The "hint" provided could imply a regressor $X=1,2,...$ which is not random, and for which obviously the stability condition is not satisfied, since the average moment "matrix" is a 2nd-degree polynomial in $n$ which does not converge.

If indeed this is the regressor implied, then regarding speed of convergence we have

\begin{align*} n^p(b-\beta) &= n^p\left(\frac 1n X'X\right)^{-1} \frac 1n X'\epsilon\\ &= \frac{6n^p}{(n+1)(2n+1)} \cdot \left(\frac 1n\sum_{i=1}^ni\epsilon_i\right)\\ &=\frac{6n^{p+1/2}}{(n+1)(2n+1)} \cdot \left(\frac 1{\sqrt n}\sum_{i=1}^n(i/n)\epsilon_i\right)\end{align*}

Now the last term converges to a zero-mean normal (see for example Hamilton's "Time-Series Analysis", the chapter about Deterministic Time Trends), while in order for the first to converge to a finite non-zero constant we need $p+1/2 = 2$ (to match orders of magnitude of numerator and denominator). So $p=3/2$ and so the speed of convergence is $n^{3/2} = n\sqrt n$.

Part 2) can be worked analogously.

The sequence $\{(i/n)\epsilon_i\}$ is a martingale difference (m.d.s) because
a) $E[(i/n)\epsilon_i] = 0$ and
b) $E[(i/n) \epsilon_i\mid \epsilon_{i-1}, \epsilon_{i-2},... ] = 0$

In order then for $\frac 1{\sqrt n}\sum_{i=1}^n(i/n)\epsilon_i$ to obey the Central Limit Theorem we need more over the following conditions (denote $\sigma^2$ the variance of $\epsilon$ and $v_i$ the variance of $(i/n)\epsilon_i$):
c) $v_i=E[(i/n)\epsilon_i]^2>0$ for all $i$
d) $\frac 1n \sum_{i=1}^nv_i \rightarrow v>0$
e) $E|(i/n)\epsilon_i|^r<\infty$ for some $r>2$ and all $i$
f) $\frac 1n \sum_{i=1}^n[(i/n)\epsilon_i]^2 \rightarrow v$

Condition c) is obviously satisfied. For condition e) assume that the 4th moment of $\epsilon$ exists and it is finite (usually an innocuous assumption related to the real world phenomenon under study), set $r=4$ and the condition is satisfied. For condition d) we have

$$\frac 1n \sum_{i=1}^nv_i = \frac 1n \sum_{i=1}^nE[(i/n)\epsilon_i]^2 = \frac {\sigma^2}{n^3} \sum_{i=1}^ni^2 = \frac {\sigma^2}{n^3}\frac{1}{6}n(n+1)(2n+1) \rightarrow \frac {\sigma^2}{3}$$

so condition d) is satisfied. I 'll leave condition f) unproven, it needs a roundabout way -but it holds, as demonstrated in the reference I have provided. Behind these technical requirements the intuition is the usual one associated with the CLT: the variance does not explode, neither does it tend to zero (although this intuition is just a step in understanding - there exists the generalized CLT for processes with infinite variance).

• Thanks a lot for your help, I really appreciate it. Could you please give me some more intuition about/ show me why the last term converges to a zero-mean normal?
– rbm
Feb 6, 2014 at 17:33
• @rbm I added something about that. Feb 6, 2014 at 18:45