3
$\begingroup$

I have the following standard linear regression model: $y_{i}=\beta_{1}+\beta_{2}x_{2,i}+\beta_{3}x_{3,i}+\varepsilon_{i}$ where $\varepsilon_{i}$ is normally distributed with mean 0 and variance $\sigma^{2}$. The $x$'s can be treated as exogenous.

Here I am not directly interested in estimating $\left(\beta_{1},\beta_{2},\beta_{3}\right)$, what I want is $\left(\beta_{1}^{2},\beta_{2}^{2},\beta_{3}^{2}\right)$ . I am aware that squaring my beta estimates will give a consistent estimator, but I am interested in something unbiased. I have a possible solution to the problem, but I am not really sure. Here it goes:

In matrix notation we have that:

$\hat{\beta} = \left(X^{'}X\right)^{-1}X'y = \left(X^{'}X\right)^{-1}X'\left(X\beta+\varepsilon\right) = \beta+\left(X^{'}X\right)^{-1}X'\varepsilon$

Define $e_{i}$ as a $1\times3$ row vector with the i'th element being 1 and the rest 0. Then:

\begin{eqnarray*} \hat{\beta}_{i}^{2} & = & \left(e_{i}\beta+e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\\ & = & \left(e_{i}\beta\right)^{2}+\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}+2e_{i}\beta e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\\ & = & \beta_{i}^{2}+\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}+2\beta_{i}e\left(X^{'}X\right)^{-1}X'\varepsilon \end{eqnarray*}

Taking the expectation yields:

\begin{eqnarray*} E\left[\hat{\beta}_{i}^{2}\right] & = & E\left[\beta_{i}^{2}\right]+E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right]+E\left[2\beta_{i}e\left(X^{'}X\right)^{-1}X'\varepsilon\right]\\ & = & \beta_{i}^{2}+E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right]+2\beta_{i}e\left(X^{'}X\right)^{-1}X'E\left[\varepsilon\right]\\ & = & \beta_{i}^{2}+E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right] \end{eqnarray*}

Here:

$E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right]$

is the small sample bias, and we have that $e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon$ is normally distributed with mean 0 and variance:

\begin{eqnarray*} Var\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right) & = & e_{i}\left(X^{'}X\right)^{-1}X'Var\left(\varepsilon\right)X\left(X^{'}X\right)^{-1}e_{i}^{'}\\ & = & \sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'} \end{eqnarray*}

Hence, we have that:

$Z=\frac{e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon}{\sqrt{\sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'}}}\sim N\left(0,1\right)$

So we can compute the expectation as:

\begin{eqnarray*} E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right] & = & E\left[\left(\sqrt{\sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'}}Z\right)^{2}\right]\\ & = & \sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'}E\left[Z^{2}\right]\\ & = & \sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'} \end{eqnarray*}

In total I could therefore estimate $\beta^{2}$ as:

$\hat{\beta^{2}}=\left(\hat{\beta}\right)^{2}-\hat{\sigma}^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'}$

It all of this correct? Do you have any references where I may read something about this stuff?

$\endgroup$
  • 2
    $\begingroup$ If the $X$ is fixed, and $\varepsilon_i$ are iid normals with zero mean and variance $\sigma^2$, then your math is ok. These are however quite strong assumptions, which do not always hold. $\endgroup$ – mpiktas Oct 6 '15 at 10:54
  • 3
    $\begingroup$ You can use bootstrapping, and then the bootstrap bias corection. $\endgroup$ – kjetil b halvorsen Oct 6 '15 at 12:51
3
$\begingroup$

Your math is correct but imho over-complicated.

Under finite sample OLS conditions, $\hat{\beta}$ is normally distributed and its variance $= (X'X)^{-1} \sigma^2$. Now for any random variable, $\text{E}(X^2)$ = $\text{E}(X)^2 + \text{Var}(X)$. Switching terms around a bit, we have your bottomline equation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.