# Estimate $\beta^{2}$ in linear regression $y_{i}=\beta_{1}+\beta_{2}x_{2,i}+\beta_{3}x_{3,i}+\varepsilon_{i}$

I have the following standard linear regression model: $y_{i}=\beta_{1}+\beta_{2}x_{2,i}+\beta_{3}x_{3,i}+\varepsilon_{i}$ where $\varepsilon_{i}$ is normally distributed with mean 0 and variance $\sigma^{2}$. The $x$'s can be treated as exogenous.

Here I am not directly interested in estimating $\left(\beta_{1},\beta_{2},\beta_{3}\right)$, what I want is $\left(\beta_{1}^{2},\beta_{2}^{2},\beta_{3}^{2}\right)$ . I am aware that squaring my beta estimates will give a consistent estimator, but I am interested in something unbiased. I have a possible solution to the problem, but I am not really sure. Here it goes:

In matrix notation we have that:

$\hat{\beta} = \left(X^{'}X\right)^{-1}X'y = \left(X^{'}X\right)^{-1}X'\left(X\beta+\varepsilon\right) = \beta+\left(X^{'}X\right)^{-1}X'\varepsilon$

Define $e_{i}$ as a $1\times3$ row vector with the i'th element being 1 and the rest 0. Then:

\begin{eqnarray*} \hat{\beta}_{i}^{2} & = & \left(e_{i}\beta+e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\\ & = & \left(e_{i}\beta\right)^{2}+\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}+2e_{i}\beta e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\\ & = & \beta_{i}^{2}+\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}+2\beta_{i}e\left(X^{'}X\right)^{-1}X'\varepsilon \end{eqnarray*}

Taking the expectation yields:

\begin{eqnarray*} E\left[\hat{\beta}_{i}^{2}\right] & = & E\left[\beta_{i}^{2}\right]+E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right]+E\left[2\beta_{i}e\left(X^{'}X\right)^{-1}X'\varepsilon\right]\\ & = & \beta_{i}^{2}+E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right]+2\beta_{i}e\left(X^{'}X\right)^{-1}X'E\left[\varepsilon\right]\\ & = & \beta_{i}^{2}+E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right] \end{eqnarray*}

Here:

$E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right]$

is the small sample bias, and we have that $e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon$ is normally distributed with mean 0 and variance:

\begin{eqnarray*} Var\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right) & = & e_{i}\left(X^{'}X\right)^{-1}X'Var\left(\varepsilon\right)X\left(X^{'}X\right)^{-1}e_{i}^{'}\\ & = & \sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'} \end{eqnarray*}

Hence, we have that:

$Z=\frac{e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon}{\sqrt{\sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'}}}\sim N\left(0,1\right)$

So we can compute the expectation as:

\begin{eqnarray*} E\left[\left(e_{i}\left(X^{'}X\right)^{-1}X'\varepsilon\right)^{2}\right] & = & E\left[\left(\sqrt{\sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'}}Z\right)^{2}\right]\\ & = & \sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'}E\left[Z^{2}\right]\\ & = & \sigma^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'} \end{eqnarray*}

In total I could therefore estimate $\beta^{2}$ as:

$\hat{\beta^{2}}=\left(\hat{\beta}\right)^{2}-\hat{\sigma}^{2}e_{i}\left(X^{'}X\right)^{-1}e_{i}^{'}$

• If the $X$ is fixed, and $\varepsilon_i$ are iid normals with zero mean and variance $\sigma^2$, then your math is ok. These are however quite strong assumptions, which do not always hold. – mpiktas Oct 6 '15 at 10:54
• You can use bootstrapping, and then the bootstrap bias corection. – kjetil b halvorsen Oct 6 '15 at 12:51

Your math is correct but imho over-complicated.

Under finite sample OLS conditions, $\hat{\beta}$ is normally distributed and its variance $= (X'X)^{-1} \sigma^2$. Now for any random variable, $\text{E}(X^2)$ = $\text{E}(X)^2 + \text{Var}(X)$. Switching terms around a bit, we have your bottomline equation.