1
$\begingroup$

Consider linear regression model $$y = X\beta + e$$

I want to construct confidence interval for fitted values, i.e. $\hat{y} = X\hat{\beta}$. Define linear fucntion of $\beta$ as $$f(\beta) = X\beta$$ and $$f'(\beta) = X$$

Now by Delta Method $$\sqrt{n}(f(\hat{\beta}) - f(\beta)) \xrightarrow{d} f'(\beta)z$$ where $z \sim N(0, V_{\beta})$

$$Var(\sqrt{n}(f(\hat{\beta}) - f(\beta))) = XV_{\beta}X^T$$ Hence $$\sqrt{n}(f(\hat{\beta}) - f(\beta)) \xrightarrow{d} N(0, XV_{\beta}X^T) $$

How do I proceed from now? I can construct 95% confidence interval for the $\sqrt{n}(f(\hat{\beta}) - f(\beta))$ but it does not give much information since I don't know $f(\beta)$. It is stated in the book by Hansen (freely available online, section 6.14) that this interval should be $$CI = x_i^T\hat{\beta} \pm 1.96\sqrt{x_i^T\hat{V}_{\hat{\beta}}x_i}$$ where $\hat{V}_{\hat{\beta}}$ is estimated var-covar matrix for $\hat{\beta}$. Probably $\sqrt{X\hat{V}_{\hat{\beta}}X^T}$ denotes standard error term.

So far I have two questions: (1) how do you come up with Hansen's formula and (2) why dimensions don't match in this formula. $X\hat{V}X^T$ should have dimension $n \times n$ since $X$ is $n\times k$ and $V$ is $k\times k$. However, $X\beta$ is $n\times 1$ matrix.

EDIT When I read my question I realized that if $f(\hat{\beta_i}) - f(\beta_i) \sim N(0, \sigma_i^2)$, then CI for $f(\beta_i)$ is given as $$CI = f(\hat{\beta}_i) \pm 1.96\sqrt{\sigma_i^2}$$

So, assuming that $n$ is big enough one could claim that $$\sqrt{n}(f(\hat{\beta}) - f(\beta)) \sim N(0, XV_{\beta}X^T)$$ But then $$(f(\hat{\beta}) - f(\beta)) \sim N(0, \frac{1}{n}XV_{\beta}X^T)$$ So, I would reformulate my first question: why there is no division by $n$ in Hansen's formula?

|cite|improve this question
$\endgroup$
  • $\begingroup$ Could you give a reference/link to the "book by Hansen"? $\endgroup$ – kjetil b halvorsen Oct 16 '16 at 19:42
  • $\begingroup$ The link is ssc.wisc.edu/~bhansen/econometrics. Actually I think that I found the answer to both questions. First one should distinct asymptotic variance of $\hat{\beta}$ and exact variance of $\hat{\beta}$. In the book the first one is denoted as $V_{\beta}$ and the latter one as $V_{\hat{\beta}}$. Using relation $n\hat{V}_{\hat{\beta}} = \hat{V}_{\beta}$ one can replace $1/nV_{\beta}$ by $V_{\hat{\beta}}$ in the last formula of the post (assuming that we use consistent estimator for variance of $\hat{\beta}$). $\endgroup$ – tosik Oct 17 '16 at 13:07
  • $\begingroup$ I also misunderstood matrix and vector notation. In the book $X$ is $n\times k$ matrix where $k$ in number of regressors and $n$ number of observations. However, when Hansen switches to notation of the form $X^TX = \sum_{i=1}^{n}x_ix_i^T$ he uses $x_i$ to denote $k\times 1$ vector which contains all independent variables for one particular observation $i$. In this case dimensions match. I edited question to avoid confusion. $\endgroup$ – tosik Oct 17 '16 at 13:17
  • $\begingroup$ @IvanovNikita, post this as an answer, so that the community knows we don't have to bother figuring this out. It does seem to me like these two thoughts do address your confusion about Hansen's notation. $\endgroup$ – StasK Oct 17 '16 at 17:11
1
$\begingroup$

So far the questions in this post arised from the confusion of notation from Hansen's book.

  1. Why there is $\hat{V}_{\hat{\beta}}$ in the final formula when it should be $\frac{1}{n}\hat{V}_{\beta}$? You should be careful with notation of variances since there are many of them (4 to be precise). $V_{\beta}$ denotes asymptotic variance of $\sqrt{n}(\hat{\beta} - \beta)$. While $V_{\hat{\beta}}$ corresponds to exact variance of $\hat{\beta}$. Finally, $\hat{V}_{\hat{\beta}}$ and $\hat{V}_{\beta}$ denote estimators of "usual" and asymptotic variances of $\hat{\beta}$ respectively. Once you observed that $\hat{V}_{\beta} = n\hat{V}_{\hat{\beta}}$ confusion why there is no division by $n$ is gone.
  2. The second question again arised because of confusion on the notation. Formula for confidence interval is given as $$CI = x_i'\hat{\beta} \pm 1.96\sqrt{x_i'\hat{V}_{\hat{\beta}}x_i}$$ where $x_i$ is not the same as $X_i$. The latter one is usual $n\times k$ matrix where each column corresponds to the regressor, while $x_i'$ denotes $i^{th}$ row of this matrix. After recognizing this fact it is obvious that dimensions match.
|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.