As you correctly note the original difference is because in the first case you use the "raw" polynomials while in the second case you use the orthogonal polynomials. Therefore if the later
lm call was altered into:
fit3<-lm(y~ poly(x,degree=2, raw = TRUE) -1) we would get the same results between
fit3. The reason why we get the same results in this case is "trivial"; we fit the exact same model as we fitted with
fit<-lm(y~.-1,data=x_exp), no surprises there.
One can easily check that the model matrices by the two models are the same
all.equal( model.matrix(fit), model.matrix(fit3) , check.attributes= FALSE) # TRUE).
What is more interesting is why you will get the same plots when using an intercept. The first thing to notice is that, when fitting a model with an intercept
In the case of
fit2 we simply move the model predictions vertically; the actual shape of the curve is the same.
On the other hand including an intercept in the case of
fit results into not only a different line in terms of vertical placement but with a whole different shape overall.
We can easily see that by simply appending the following fits on the existing plot.
lines(xp,yp, col='green', lwd = 2)
fit2_b<-lm(y~ poly(x,degree=2, raw = FALSE) )
OK... Why were the no-intercept fits different while the intercept-including fits are the same? The catch is once again on the orthogonality condition.
In the case of
fit_b the model matrix used contains non-orthogonal elements, the Gram matrix
crossprod( model.matrix(fit_b) ) is far from diagonal; in the case of
fit2_b the elements are orthogonal (
crossprod( model.matrix(fit2_b) ) is effectively diagonal).
As such in the case of
fit when we expand it to include an intercept in
fit_b we changed the off-diagonal entries of the Gram matrix $X^TX$ and thus the resulting fit is different as a whole (different curvature, intercept, etc.) in comparison with the fit provided by
fit. In the case of
fit2 though when we expand it to include an intercept as in
fit2_b we only append a column that is already orthogonal to the columns we had, the orthogonality is against the constant polynomial of degree 0. This simply results on vertically moving our fitted line by the intercept. This is why the plots are different.
The interesting by-question is why the
fit2_b are the same; after all the model matrices from
fit2_b are not the same in face value. Here we just need to remember that ultimately
fit2_b have the same information.
fit2_b is just a linear combination of the
fit_b so essentially their resulting fits will be the same. The differences observed in the fitted coefficient reflects the linear recombination of the values of
fit_b in order to get them orthogonal. (see G. Grothendieck answer here too for different example.)