# what does model complexity means in linear regression?

As far as I know, $$y=\beta x$$ is a not a complex model since we have a polynomial of the first order for all variables $$x_i$$. I am studying the linear the bias variance trade-off, and the lecture mentioned that we can reduce the complexity of the model by reducing the number of variables. This made me confused with my initial intuition about the definition of the complexity of the model. Can any one explain to me how the number of variable can make the above model more complex and how it could go through all the data points while it is only a first grade model? Thanks

• The general idea is that you want your model to has a few variables/terms as possible (principle of parsimony). The fewer terms you have, the easier it is for someone to interpret your model. You're also right in your thinking by the way - adding polynomial terms higher than degree one leads to an increase in model complexity. In short, model complexity increases as you add more variables, or variables with some sort of transformation ($x^n$, $\log(x)$, etc.). Apr 24, 2022 at 5:34
• @ralph Would you want to post that as an answer?
– Dave
Apr 24, 2022 at 5:54
• Parsimony is the enemy of predictive discrimination. Parsimony is not how the world works. And the data do not possess sufficient information content to allow one to determine which variables are the "big predictors". Apr 24, 2022 at 12:05

In short, model complexity increases as you add more variables, or variables with some sort of transformation ($$x^n$$, $$\log(x)$$, etc.).