2
$\begingroup$

Consider the DGP $y_i=x_i+\epsilon_i$, where $\epsilon_i \sim Z$. We estimate $\beta=1$ by regression without a constant term, so in $y_i=\beta x_i + \epsilon_i$.

  1. Show that this DGP does not satisfy the stability assumption ($\operatorname{plim}(\frac{1}{n}X'X) \rightarrow Q, Q$ invertible) and show that the speed of convergence of $b$ to $\beta$ is $n\sqrt{n}$ in this case. (Hint: $\sum_{i=1}^n i^2=\frac{1}{6}n(n+1)(2n+1))$
  2. Let $x_i=\frac{1}{i}$. Show that again the stability assumption is not satisfied and that the speed of convergence is $n^0$. (Hint: $\sum_{i=1}^\infty \frac{1}{i^2}=\frac{1}{6}\pi^2$).

What I tried for (1): $\frac{1}{n}x'x=\frac{1}{n}\sum x_i^2=\frac{1}{6}(n+1)(2n+1)$. Which does not satisfy the assumption I guess?

Also \begin{align*} n^p(b-\beta) &= (x'x)^{-1} x'\epsilon\\ &= \left(\frac{1}{n^p} x'x\right)^{-1} x'\epsilon\\ &= \frac{6n^p}{n(n+1)(2n+1)} x'\epsilon\end{align*}

And I think that I need to show that if I would take the limit $n\rightarrow \infty$, the first term would evaluate to a constant if $p=\frac{3}{2}$. I don't however see why this is so ($p=3$ would follow from my derivation I believe).

What I tried for (2). $\frac{1}{n}x'x=\frac{1}{n}\sum_{i=1}^n \frac{1}{i^2}$. Again this doesn't satisfy the assumption? And I'm not sure about the second part of this subquestion.

Thank you in advance for your help :)!

$\endgroup$
  • $\begingroup$ You forgot to provide the most important part: what are $x_i$'s. They are random variables alright, but what are their characteristics? Do they form a stationary series? What? It all depends on that information. And what does $Z$ stands for? $\endgroup$ – Alecos Papadopoulos Feb 5 '14 at 20:09
  • 1
    $\begingroup$ Please supply missing details. "DGP" = discrete Gaussian process? Differentiable Gaussian process? Data generating process? Something else? And what does "$\tilde Z$" mean? What is $b$--the ordinarily least squares estimator of $\beta$? And what is the connection between $X$ (a random matrix, evidently) and $x$ (a determined, non-random matrix)? $\endgroup$ – whuber Feb 5 '14 at 20:12
  • $\begingroup$ First of all, sorry for using a temporary account but I had difficulty logging in to my own account so I was the one to ask the question. $\endgroup$ – rbm Feb 6 '14 at 7:48
  • $\begingroup$ @alecos That was also what I found confusing, since it is not mentioned in the question. But I think that there are just $n \ x_i$ 's that together form the vector $x$ but I am not sure. Z is the standard normal distribution. $\endgroup$ – rbm Feb 6 '14 at 7:50
  • $\begingroup$ @whuber DGP is data generating process. Z is the standard normal distribution. b is indeed the OLS as you mentioned. And I let $x$ denote a random vector of the $x_i$'s, but I'm not sure whether my notation is correct. $\endgroup$ – rbm Feb 6 '14 at 7:54
5
$\begingroup$

The original question appears ill-posed, regarding part 1). For arbitrary regressor $X$ the stability condition may or may not be satisfied. The "hint" provided could imply a regressor $X=1,2,...$ which is not random, and for which obviously the stability condition is not satisfied, since the average moment "matrix" is a 2nd-degree polynomial in $n$ which does not converge.

If indeed this is the regressor implied, then regarding speed of convergence we have

\begin{align*} n^p(b-\beta) &= n^p\left(\frac 1n X'X\right)^{-1} \frac 1n X'\epsilon\\ &= \frac{6n^p}{(n+1)(2n+1)} \cdot \left(\frac 1n\sum_{i=1}^ni\epsilon_i\right)\\ &=\frac{6n^{p+1/2}}{(n+1)(2n+1)} \cdot \left(\frac 1{\sqrt n}\sum_{i=1}^n(i/n)\epsilon_i\right)\end{align*}

Now the last term converges to a zero-mean normal (see for example Hamilton's "Time-Series Analysis", the chapter about Deterministic Time Trends), while in order for the first to converge to a finite non-zero constant we need $p+1/2 = 2$ (to match orders of magnitude of numerator and denominator). So $p=3/2$ and so the speed of convergence is $n^{3/2} = n\sqrt n$.

Part 2) can be worked analogously.

ADDENDUM
The sequence $\{(i/n)\epsilon_i\}$ is a martingale difference (m.d.s) because
a) $E[(i/n)\epsilon_i] = 0 $ and
b) $E[(i/n) \epsilon_i\mid \epsilon_{i-1}, \epsilon_{i-2},... ] = 0$

In order then for $\frac 1{\sqrt n}\sum_{i=1}^n(i/n)\epsilon_i$ to obey the Central Limit Theorem we need more over the following conditions (denote $\sigma^2$ the variance of $\epsilon$ and $v_i$ the variance of $(i/n)\epsilon_i$):
c) $v_i=E[(i/n)\epsilon_i]^2>0$ for all $i$
d) $\frac 1n \sum_{i=1}^nv_i \rightarrow v>0$
e) $E|(i/n)\epsilon_i|^r<\infty$ for some $r>2$ and all $i$
f) $\frac 1n \sum_{i=1}^n[(i/n)\epsilon_i]^2 \rightarrow v$

Condition c) is obviously satisfied. For condition e) assume that the 4th moment of $\epsilon$ exists and it is finite (usually an innocuous assumption related to the real world phenomenon under study), set $r=4$ and the condition is satisfied. For condition d) we have

$$\frac 1n \sum_{i=1}^nv_i = \frac 1n \sum_{i=1}^nE[(i/n)\epsilon_i]^2 = \frac {\sigma^2}{n^3} \sum_{i=1}^ni^2 = \frac {\sigma^2}{n^3}\frac{1}{6}n(n+1)(2n+1) \rightarrow \frac {\sigma^2}{3}$$

so condition d) is satisfied. I 'll leave condition f) unproven, it needs a roundabout way -but it holds, as demonstrated in the reference I have provided. Behind these technical requirements the intuition is the usual one associated with the CLT: the variance does not explode, neither does it tend to zero (although this intuition is just a step in understanding - there exists the generalized CLT for processes with infinite variance).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks a lot for your help, I really appreciate it. Could you please give me some more intuition about/ show me why the last term converges to a zero-mean normal? $\endgroup$ – rbm Feb 6 '14 at 17:33
  • $\begingroup$ @rbm I added something about that. $\endgroup$ – Alecos Papadopoulos Feb 6 '14 at 18:45
  • $\begingroup$ Thanks a lot for your help. I would accept your answer if I had access to the other account but I don't. Could a moderator maybe accept this answer for me? $\endgroup$ – rbm Feb 7 '14 at 13:43
  • $\begingroup$ @rbm Don't mind about that. $\endgroup$ – Alecos Papadopoulos Feb 7 '14 at 17:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.