I am working through a problem where it requires me to derive a least squares estimator for the model $Y_i = \beta_1X_i + \varepsilon_i$. I thought that the estimated $\beta_1$ would simply be the sample mean of Y over the sample mean of X(since the regression function passes through the origin and the mean of the two variables). Writing out the cost function, I get $F = \sum_{i=1}^{n} (Y_i-\beta_1X_i)^2$. After deriving this and setting it equal to zero, I get that the minimum betaone is achieved when $\Sigma X_iY_i = \beta_1\Sigma X_i^2$, implyig that $\widehat\beta_1 = \frac{\Sigma X_iY_i}{\Sigma X_i^2}$, which does not appear to be the ratio of the sample means. Have I made a mistake or do I have some sort of misunderstanding here?
$\begingroup$
$\endgroup$
-
1$\begingroup$ What happens when the sample mean of the $X_i$ is zero? Indeed, in many applications the $X_i$ are first standardized in order to make this condition hold! $\endgroup$ – whuber♦ Feb 1 '18 at 21:53
-
$\begingroup$ Is the $Y$ mean-centered or is this a regression-through-the-origin problem? If so, the expression you have for $\hat{\beta}$ as $\sum X_i Y_i / \sum X_i^2$ isn't right because that's the expression for regression with an intercept term... $\endgroup$ – AdamO Feb 1 '18 at 23:21
-
$\begingroup$ The assumption in the model is that $\beta_0 = 0$ so that the model should pass through the origin. $\endgroup$ – GTOgod Feb 2 '18 at 0:48
$\begingroup$
$\endgroup$
Your initial guess was not wrong. It simply corresponds to an estimator with a different constraint.
- For the ordinary least-squares estimator, you want to minimise the squared-error so you want: $\frac{\partial(Y-\beta X)^T(Y-\beta X)}{\partial \beta}=0$, which gives you $\beta=\frac{X^TY}{X^TX}$
- If instead, you simply had the constraint that your estimator is unbiased (on average) and nothing more, then you want: $E(Y-\beta X)=0$, which gives you $\beta=\frac{E(Y)}{E(X)}$
However, as @whuber wrote in the comment, the second one is likely to fail in practice so it is extremely unpopular.
