Given two linear regression models, which model would perform better? I have taken up a machine learning course at my college. In one of the quizes, this question was asked.

Model 1 : $$ y = \theta x + \epsilon $$
Model 2 : $$ y = \theta x + \theta^2 x + \epsilon $$
Which of the above models would fit data better? (assume data can be modelled using linear regression)

The correct answer (according to the professor) is that both models would perform equally well. However I believe that the first model would be a better fit.
This is the reason behind my answer. The second model, which can be rewritten as $ \alpha x + \epsilon $, $\alpha = \theta + \theta^2$ would not be the same as the first model. $\alpha$ is in fact a parabola, and hence has a minimum value ($ -0.25 $ in this case). Now because of this, the range of $ \theta $ in the first model is greater than the range of $ \alpha $ in the second model. Hence if the data was such that the best fit had a slope less than $-0.25$, the second model would perform very poorly as compared to the first one. However in case the slope of the best fit was greater than $-0.25$, both models would perform equally well.
So is the first one better, or are both the exact same?
 A: Model 2 can be written as: 
$$y=(\theta + \theta^{2}) x+\epsilon=\beta x+\epsilon.$$
This seems similar to model 1, just with different notation for the hyperparameters ($\theta, \beta $). However, for model 1 we can write
$$\hat{\theta}=(X^{'}X)^{-1}X^{'}y.$$
But since in model 2 we have that 
$$\beta=\theta + \theta^{2},$$
then as you mentioned indeed the range of $\hat{\beta}$ should belong to $[-0.25,+\infty]$ for $\theta \in R$. Which will lead to difference in these 2 models.
Thus in model 2 you are constraining your coefficient estimate unlike model 1.
To make this more clear, it should be noted that in model 1, $\hat{\theta}$ is obtained through minimizing the square loss function
$$\hat{\theta}=\arg\min_{\theta\in{R}}    \ \ (y-X\theta)^{'}(y-X\theta)=(X^{'}X)^{-1}X^{'}y.$$
However in the model 2 the estimate is obtained through 
$$\hat{\beta}=\arg\min_{\beta\geq-0.25}    \ \ (y-X\beta)^{'}(y-X\beta)$$
 which might lead to a different result.
A: Not sure I understand your reasoning. If you take:
$$y = \alpha x+\epsilon$$ and $$y = \theta x + \epsilon$$
and estimate $\alpha$ and $\theta$ using a simple linear regression, you will get $\alpha$=$\theta$. Moreover, since the methodology is exactly the same there is no difference in the $R^2$ value you would get in either equation. The underlying value of $\theta$ in the first equation will of course be different, since $\alpha = \theta + \theta^2$, but this has nothing to do with fit.
