# Understanding concepts of regularization

I am trying to understand regularization in machine learning. But, I do not understand some fundamental concepts in this topic, could you please explain?

1. A model that has high variance, captures noise/randomness in the data, this translates to an increase in coefficients. Why does a high variance model, say: $$y_h$$ have the value of coefficients greater than a low variance model, say: $$y_l$$ ? An example in linear regression model according to what I interpret is:
$$y_h = 2 + 13x_1 + 51x_2$$
$$y_l = 2 + 3x_1 + 5x_2$$ Is this interpretation correct? Could you explain this increase of coefficients with noise?
2. Continuing with linear regression example, to minimize the coefficients which capture noise, a shrinkage quantity is added to the RSS (residual sum of squares) in ridge regression
$$\sum_{i=1}^n \left( y_i - \beta_0 - \sum_{j=1}^p \beta_j x_{ij} \right) + \lambda \sum_{j=1}^p \beta_j^2$$ = $$RSS + Shrinkage$$
the $$\beta_j$$ in the RSS is not alone but accompanied by the $$x_j$$, then why is $$\beta^2$$ added to RSS and not $$(x_j \beta_j)^2$$ ?
3. Why is the intercept $$\beta_0$$ not shrunk?

1. If the predicted variable ($$y$$) is noisy, then, it increases the variance in the coefficient estimates. What this means is that our coefficient estimate will vary quite a bit if we draw different samples from the population. This is the variance problem which people often talk about. This problem is exacerbated if you have too many independent variables in the linear regression (hence a very flexible model which starts fitting the sample noise in y).
2. If the predictor variables ($$x$$) are noisy, then, it would lead to consistently smaller estimates of the coefficients. This is called attenuation bias.
The Intercept parameter is set by the constraint that our prediction, $$\hat{y}$$ should have the same mean as $$y$$. It's not affected by a noise which has a mean$$= 0$$. That's why we don't regularize it.