Currently learning ridge regression and I was a little confused about the penalisation of more complex models (or the definition of a more complex model).
From what I understand, model complexity doesn't necessarily correlate to the polynomial order. So: $$ 2 + 3+ 4x^2 + 5x^3 + 6x^4$$ is a more complex model than: $$ 5x^5$$
And I know that the point of regularisation is to keep the model complexity low, so say for example we have a 5th order polynomial $$ f(x; w) = w_0 + w_1x + w_2x^2 + w_3x^3 + w_4x^4 + w_5x^5$$
The more parameters that are 0 the better.
But what I don't understand is, if it was the same order polynomial why do lower parameter values get penalised less? So why would:
$$ 2 + 5x + x^3$$ be a less complex model than
$$ 433+ 342x + 323x^3$$ they are both of the same polynomial order, and the parameter values just simply dependent on the data.