# Howe to perform ridge regression only on a subset of the variables

I am trying to code some algorithm that performs ridge-regression with penalty parameter $$\lambda$$ on all features except for a specific subset.

Let $$\mathbf{X}$$ be the $$n \times p$$ matrix for $$n$$ samples and $$p$$ features, $$\mathbb{y}$$ is our vector with the dependent variable for each sample.

If we applied ridge regression with penalization to ALL features, we would have our fit:

$$\mathbf{\hat{y} = X \hat{\beta} = X(X^T X+\lambda I)^{-1}Xy}$$

Nevertheless, what are you supposed to do if you don't want to penalize, say the second feature $$p = 2$$? Do we just instead of having a constant $$\lambda$$, let $$\lambda^*$$ be the penalty for the variables we want to penalize, and then

$$\lambda_i = \begin{cases}\lambda^*, & i \neq 2 \\ 0, & i = 2\end{cases}$$?

Or which way is it supposed to work?

Formally you could say this uses $$\lambda^*$$ where $$\lambda^* = a\lambda$$ and $$a$$ is a vector of penalty factors.