I have taken up a machine learning course at my college. In one of the quizes, this question was asked.
Model 1 : $$ y = \theta x + \epsilon $$ Model 2 : $$ y = \theta x + \theta^2 x + \epsilon $$
Which of the above models would fit data better? (assume data can be modelled using linear regression)
The correct answer (according to the professor) is that both models would perform equally well. However I believe that the first model would be a better fit.
This is the reason behind my answer. The second model, which can be rewritten as $ \alpha x + \epsilon $, $\alpha = \theta + \theta^2$ would not be the same as the first model. $\alpha$ is in fact a parabola, and hence has a minimum value ($ -0.25 $ in this case). Now because of this, the range of $ \theta $ in the first model is greater than the range of $ \alpha $ in the second model. Hence if the data was such that the best fit had a slope less than $-0.25$, the second model would perform very poorly as compared to the first one. However in case the slope of the best fit was greater than $-0.25$, both models would perform equally well.
So is the first one better, or are both the exact same?