Confusion regarding Least Square/Maximum Likelihood estimators under a restricted intercept? I'm a bit confused regarding how intercept restrictions impact slope estimators in a simple linear regression. An example:
$$H_0:\beta_0=0$$
$$H_A:\beta_0\neq 0$$
I simulated a variable $y_i$ as:
$$y_i= \beta_0 + \beta x_i +\epsilon_i \quad \cdots(1)\quad\quad  \beta_0\neq0$$
$$y_i= 0 + \beta x_i +\epsilon_i \quad \quad \cdots(2)$$
fitting a simple linear regression on each of the simulations yields the exact same estimators for $\beta$ under both least squares and maximum likelihood.
Now if I fit simple linear regressions while including the restriction on the intercept I get two different estimators for each of the possible simulated variables. Can someone help me understand what's going on?
What confuses me the most is the fact that the restricted and unrestricted slope estimators are not the same even when the true model for $y_i$ has no intercept.
I'm attaching my R code for reference:
x<-rnorm(100) 
err<-5*rnorm(100)
y1 <- 5*x+err
y2 <- 5+5*x+err
summary(lm(y1~x))
summary(lm(y2~x))
summary(lm(y1~x+0))
summary(lm(y2~x+0))
 

 A: I think the slope in a regression with unrestricted intercept is not influenced by the intercept. It's determined solely by the deviance of x and y over their means. As a consequence, the information of position of y (the intercept) is not influencing the slope, which is calculated in a second step.
This can be seen in the formula for the least squares estimation of the slope:
$$\hat{\beta} = \frac{\sum_i(x_i-\bar{x})(y_i-\bar{y})}{\sum_i(x_i-\bar{x})^2}$$
As you can see, all y values are subtracted from its mean, which means that this value would not change if you add +5 to all y points.
On the other hand, a restricted regression has no concept of intercept. It calculates the the slope as:
$$ \hat{\beta} = \frac{\sum_i x_iy_i}{\sum_i x^2} $$
This assumes that the intercept passes through 0. If it does not, the intercept is subtracted from all y. In that case subtracting the 5 will influence the values of y. So results will differ especially if the sample has a different optimal intercept than the one you restricted data to.
A: When you tell lm not to include an intercept, you force the regression line to go through your cloud of points, probably somewhere near $(\bar{x},\bar{y})$, and the origin. The odds are low that the slope of such a line is the same as what you would fit to just the cloud of points.
If the $y$-intercept was positive in the unconstrained linear model, forcing it to 0 will require the slope coefficient to be more positive. If the $y$-intercept was negative, forcing it to 0 will yield a more negative slope.
In general, there are almost never good reasons to suppress the intercept. Even if you expect the intercept to be 0, allowing for a non-zero value lets you correct for imbalances or trends in the data -- which you might not expect to continue in the future.
