From The Elements of Statistical Learning (2nd Edition; pages 63-64):

The ridge solutions are not equivariant under scaling of the inputs, and so one normally standardizes the inputs before solving (3.41). In addition, notice that the intercept $\beta_0$ has been left out of the penalty term. Penalization of the intercept would make the procedure depend on the origin chosen for $Y$; that is adding a constant $c$ to each of the targets $y_i$ wold not simply result in a shift of the predictions by the same amount $c$. ... The solution adds a positive constant to the diagonal of $\mathbf{X}^T\mathbf{X}$ before inversion. This makes the problem nonsingular, even if $\mathbf{X}^T\mathbf{X}$ is not of full rank, and was the main motivation for ridge regression when it was first introduced in statistics (Hoerl and Kennard, 1970).

Hastie et al. go on to write:

Ridge regression can also be derived as the mean or mode of a posterior distribution, with a suitably chosen prior distribution. In detail, suppose $y_i \sim \mathcal{N}(\beta_0 + x_i^T\beta, \sigma^2)$, and the parameters $\beta_j$ are each distributed as $\mathcal{N}(0, \tau^2)$, independently of one another. ... Thus the ridge regression estimate is the mode of the posterior distribution; since the distribution is Gaussian, it is also the posterior mean.

Since the intercept doesn't have the same rules applied to it as the other coefficients, $\mathbf{X}$ doesn't need to be full-rank, and the Bayesian perspective reminds me of the approach one should take with varying-intercept hierarchical regression models – where the intercepts $\alpha_j[i]$ are centered around $\beta_0$ (the non-regularized intercept), effectively – it sounds to me like one-hot encoding would be the way to go here.

From The Elements of Statistical Learning (2nd Edition; pages 63-64):

The ridge solutions are not equivariant under scaling of the inputs, and so one normally standardizes the inputs before solving (3.41). In addition, notice that the intercept $\beta_0$ has been left out of the penalty term. Penalization of the intercept would make the procedure depend on the origin chosen for $Y$; that is adding a constant $c$ to each of the targets $y_i$ wold not simply result in a shift of the predictions by the same amount $c$. ... The solution adds a positive constant to the diagonal of $\mathbf{X}^T\mathbf{X}$ before inversion. This makes the problem nonsingular, even if $\mathbf{X}^T\mathbf{X}$ is not of full rank, and was the main motivation for ridge regression when it was first introduced in statistics (Hoerl and Kennard, 1970).

Hastie et al. go on to write:

Ridge regression can also be derived as the mean or mode of a posterior distribution, with a suitably chosen prior distribution. In detail, suppose $y_i \sim \mathcal{N}(\beta_0 + x_i^T\beta, \sigma^2)$, and the parameters $\beta_j$ are each distributed as $\mathcal{N}(0, \tau^2)$, independently of one another. ... Thus the ridge regression estimate is the mode of the posterior distribution; since the distribution is Gaussian, it is also the posterior mean.

Since the intercept doesn't have the same rules applied to it as the other coefficients, $\mathbf{X}$ doesn't need to be full-rank, and the Bayesian perspective reminds me of an approach one could take with varying-intercept hierarchical regression models – where the intercepts $\alpha_{j[i]}$ are adjustments from a global intercept -- it sounds to me like one-hot encoding would be the way to go here.

From The Elements of Statistical Learning (2nd Edition; pages 63-64):

The ridge solutions are not equivariant under scaling of the inputs, and so one normally standardizes the inputs before solving (3.41). In addition, notice that the intercept $\beta_0$ has been left out of the penalty term. Penalization of the intercept would make the procedure depend on the origin chosen for $Y$; that is adding a constant $c$ to each of the targets $y_i$ wold not simply result in a shift of the predictions by the same amount $c$. ... The solution adds a positive constant to the diagonal of $\mathbf{X}^T\mathbf{X}$ before inversion. This makes the problem nonsingular, even if $\mathbf{X}^T\mathbf{X}$ is not of full rank, and was the main motivation for ridge regression when it was first introduced in statistics (Hoerl and Kennard, 1970).

Hastie et al. go on to write:

Ridge regression can also be derived as the mean or mode of a posterior distribution, with a suitably chosen prior distribution. In detail, suppose $y_i \sim \mathcal{N}(\beta_0 + x_i^T\beta, \sigma^2)$, and the parameters $\beta_j$ are each distributed as $\mathcal{N}(0, \tau^2)$, independently of one another. ... Thus the ridge regression estimate is the mode of the posterior distribution; since the distribution is Gaussian, it is also the posterior mean.

Since the intercept doesn't have the same rules applied to it as the other coefficients, $\mathbf{X}$ doesn't need to be full-rank, and the Bayesian perspective reminds me of the approach one should take with varying-intercept hierarchical regression models, it sounds to me like one-hot encoding would be the way to go here.

From The Elements of Statistical Learning (2nd Edition; pages 63-64):

The ridge solutions are not equivariant under scaling of the inputs, and so one normally standardizes the inputs before solving (3.41). In addition, notice that the intercept $\beta_0$ has been left out of the penalty term. Penalization of the intercept would make the procedure depend on the origin chosen for $Y$; that is adding a constant $c$ to each of the targets $y_i$ wold not simply result in a shift of the predictions by the same amount $c$. ... The solution adds a positive constant to the diagonal of $\mathbf{X}^T\mathbf{X}$ before inversion. This makes the problem nonsingular, even if $\mathbf{X}^T\mathbf{X}$ is not of full rank, and was the main motivation for ridge regression when it was first introduced in statistics (Hoerl and Kennard, 1970).

Hastie et al. go on to write:

Ridge regression can also be derived as the mean or mode of a posterior distribution, with a suitably chosen prior distribution. In detail, suppose $y_i \sim \mathcal{N}(\beta_0 + x_i^T\beta, \sigma^2)$, and the parameters $\beta_j$ are each distributed as $\mathcal{N}(0, \tau^2)$, independently of one another. ... Thus the ridge regression estimate is the mode of the posterior distribution; since the distribution is Gaussian, it is also the posterior mean.

Since the intercept doesn't have the same rules applied to it as the other coefficients, $\mathbf{X}$ doesn't need to be full-rank, and the Bayesian perspective reminds me of the approach one should take with varying-intercept hierarchical regression models – where the intercepts $\alpha_j[i]$ are centered around $\beta_0$ (the non-regularized intercept), effectively – it sounds to me like one-hot encoding would be the way to go here.