OLS estimate of $\beta$

Question

The random variables $Y_i$, $i = 1,...,n$ are normally and independently distributed with $E(Y_i) = \beta x_i$ and $Var(Y_i) = \sigma^2 \times x_i$. Find the ordinary least squares estimate of $\beta$. What are the expected value and variance of this estimate?

Answer 1

The OLS estimator in linear regression has a standard matrix expression that is well-known. In the case of a model with no intercept term and a single explanatory variable, you have a single vector of explanatory values $\boldsymbol{x}$ and a vector of response values $\boldsymbol{y}$. The OLS coefficient estimator is:

$$\hat{\beta}_1 = (\boldsymbol{x}^\text{T} \boldsymbol{x})^{-1} (\boldsymbol{x}^\text{T} \boldsymbol{y}) = \cdots = \sum_{i=1}^n f(x_i) y_i,$$

where $f$ is some function you can figure out. See if you can simplify the matrix expression into scalar form to get your estimator. Once you have done that, you have the estimator as a linear function of the response values, and so you can get the expected value and variance of the estimator using standard rules for linear functions:

$$\mathbb{E}(\hat{\beta}_1) = \sum_{i=1}^n f(x_i) \mathbb{E}(Y_i) = \cdots = f(\boldsymbol{x}, \beta_1),$$

$$\mathbb{V}(\hat{\beta}_1) = \sum_{I=1}^n f(x_i)^2 \mathbb{V}(Y_i) = \cdots = f(\boldsymbol{x}, \sigma).$$

Once you have got the results, have a look to see if the estimator is biased (i.e., is its expected value equal to the thing it is used to estimate) and also have a look to see how the different data values impact the variance. This is a problem where you have heteroscedasticity, but you are using the OLS estimator despite that, so it is a good exercise to see the consequences of this kind of misspecification of the estimator. Let us know how you go.