Ridge Regression: changing tuning parameter

Question

I know for ridge regression, we add a penalty term with a fixed constant λ for all β the as tuning parameter. Just wonder why choosing a fixed λ instead of varying its value for different β?

Like this:

Answer 1

If you varied the penalty for every $\beta$ the effective number of parameters would be $2\times p$ and you would have overfitting once again. Essentially the model would be non-identifiable. Penalization works by borrowing information or tilting estimates towards a preconceived bias in a systematic way. If you don't connect the $\beta$s in some way you are not borrowing information. What would work would be to group the $X$s using prior knowledge about the likely importance of them, and to give a different $\lambda$ to each group, assuming every group has $> 1$ predictor in it. The pre-grouping would not be allowed to use $Y$.