I am working through a problem where it requires me to derive a least squares estimator for the model $Y_i = \beta_1X_i + \varepsilon_i$. I thought that the estimated $\beta_1$ would simply be the sample mean of Y over the sample mean of X(since the regression function passes through the origin and the mean of the two variables). Writing out the cost function, I get $F = \sum_{i=1}^{n} (Y_i-\beta_1X_i)^2$. After deriving this and setting it equal to zero, I get that the minimum betaone is achieved when $\Sigma X_iY_i = \beta_1\Sigma X_i^2$, implyig that $\widehat\beta_1 = \frac{\Sigma X_iY_i}{\Sigma X_i^2}$, which does not appear to be the ratio of the sample means. Have I made a mistake or do I have some sort of misunderstanding here?

  • 1
    $\begingroup$ What happens when the sample mean of the $X_i$ is zero? Indeed, in many applications the $X_i$ are first standardized in order to make this condition hold! $\endgroup$
    – whuber
    Feb 1 '18 at 21:53
  • $\begingroup$ Is the $Y$ mean-centered or is this a regression-through-the-origin problem? If so, the expression you have for $\hat{\beta}$ as $\sum X_i Y_i / \sum X_i^2$ isn't right because that's the expression for regression with an intercept term... $\endgroup$
    – AdamO
    Feb 1 '18 at 23:21
  • $\begingroup$ The assumption in the model is that $\beta_0 = 0$ so that the model should pass through the origin. $\endgroup$
    – GTOgod
    Feb 2 '18 at 0:48

Your initial guess was not wrong. It simply corresponds to an estimator with a different constraint.

  • For the ordinary least-squares estimator, you want to minimise the squared-error so you want: $\frac{\partial(Y-\beta X)^T(Y-\beta X)}{\partial \beta}=0$, which gives you $\beta=\frac{X^TY}{X^TX}$
  • If instead, you simply had the constraint that your estimator is unbiased (on average) and nothing more, then you want: $E(Y-\beta X)=0$, which gives you $\beta=\frac{E(Y)}{E(X)}$

However, as @whuber wrote in the comment, the second one is likely to fail in practice so it is extremely unpopular.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.