# Estimators - OLS & other

Question:

I am assigned this problem, but I don't quite understand what it's referring to. Is $\alpha$ the same as the intercept of the regression, and the coefficient on the variable u hat is 1? Would the OLS estimator for $\alpha$ come from the normal equations, then? Why do we need to square u hat in part b - what could this be related to? I'm not sure how I can compute anything just given this information. Thank you.

This would be done just like you would estimate the intercept term in a simple regression model of the form y=α+βx+u$_i$. where you replace the least squares estimate for β by 0 in the formula for the intercept. You could also get the answer by writing down the equation for the sum of squared residuals and differentiating the function with respect to α and setting it equal to 0.

For the second part once you derive the formula you will notice that it looks like a familiar estimate for a well known parameter of a distribution.

The question does not seem to define u^$_i$. But apparently it should be y$_i$-α^ where α^ is the maximum likelihood estimate of α. Then the formula in b should look like another familiar estimate of a familiar parameter of a distribution.

• For the second part, the sum of errors looks a lot like RSS to me, but why must we divide by n-1? Does this have something to do with degrees of freedom?
– user14386
Sep 26 '12 at 20:05
• It is like an unbiased estimatro for the variance of the ys. Does that help you see what the estimate of alpha looks like? You may recall that an unbiased estimator for a variance from a distribution involves dividing by n-1 because the mean is estimated. In terms of the chi square distribution that is applied when the data are normally distributed that chi square has n-1 degrees of freedom. So you are right to think that the value of n-1 is related to degrees of freedom under the normality assumption. Sep 26 '12 at 20:16
• This is a good answer and looks very helpful. It does seem a little strange, though, to be invoking MLEs in a context that inherently is distribution-free (or, at a minimum, assumes the Gauss Markov Theorem, which likely has not yet been introduced) and where (in part b) the estimator is not an MLE.
– whuber
Sep 26 '12 at 20:52
• I think the model was intended to look like regression without a covariate which just makes the mle for alpha the sample mean for the ys and the residual variance estimate is just the sample variance of the ys. Without a covariate it is not regression. i should have referred to the estimate of beta as the least squares estimate and not the mle because the error term was not assumed to be normal and the properties of the estimatoe for beta is not relevant to the problem. I am editing the answer. Sep 27 '12 at 2:45
• It is the mean because the criterion was least sum of squared errors. If it were the smallest sum of absolute errors it would be the median. Sep 27 '12 at 17:38