Data $(x_1, y_1), \dots, (x_n, y_n)$ is modelled with $x_i$ being non random and $y_i$ being observed values of $$Y_i = \alpha + \beta (x_i - \bar x) + \sigma \epsilon_i$$ with $\epsilon_i \sim N(0,1)$.

I have calculated the MLE's of the parameters $\alpha, \beta$ and $\sigma^2$ with $\hat \alpha = \bar y$ and $$\hat \beta = \frac{\sum y_i(x_i - \bar x)}{\sum (x_i - \bar x)^2}$$ we now have a corresponding estimator for $\beta$ given by $$B = \frac{\sum Y_i(x_i - \bar x)}{\sum (x_i - \bar x)^2}$$ which apparently (according to my notes) is normally distributed with a mean of $\beta$ and a variance of $$\sigma^2 \over \sum(x_i - \bar x)^2$$ and I can't justify to myself why this is the case! Could someone please help explain this?

  • $\begingroup$ As this is basically standard bookwork, please add the self-study tag. You might like to read its tag wiki info. $\endgroup$ – Glen_b Jun 17 '13 at 0:17

Step 1: General: Recognize that $(x_i - \bar x)$, and its square, and $\frac{1}{\sum (x_i - \bar x)^2}$ are constants, not random variables. Further note that $\alpha$ and $\beta$ and $\sigma$ are also constants. First write $B$ as a constant times an expression containing a random variable. Focus carefully on the part that's not just a constant and then deal with that constant after you have that part worked out.

Step 2: Expectation: Remember that the expectation of a sum is the sum of expectations. Notice that you can take the expectation inside the summation and then move $(x_i - \bar x)$ outside that expectation. Notice you have another expression for $Y_i$ that you can use. Look again at step 1 and split up the expectation and pull out constants in the appropriate way. What's left is trivial to find the expectation of.

Step 3: Variance: Since the $Y$s are independent, the variance of a sum is the sum of the variances. You should also know a fact about the variance of a constant times a random variable to use here (more than once).

It's really nothing more than the basic properties of expectation and variance and using the facts already there in your question.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.