Coordinate Descent Alternating between LASSO and Ridge

Is there a way to do Coordinate descent but depending on the variable change the method applied to find the coefficient?

For example, apply a LASSO constraint to a predefined 3 variables and Ridge to the other 5? Or is this just elasticnet with a ratio parameter from 1 (meaning LASSO) to 0 (meaning Ridge) depending on the variable?

Main purpose is to be way more aggressive towards certain variables and shrink them to 0 while keeping others more intact.

edit

I found this code to do LASSO with CD, could I just change the beta calculation in the loop or are there more complications with the CD algo?

def lasso_nb(X, y, alpha, tol=0.00001, maxiter=50000):
n, p = X.shape
beta = np.zeros(p)
R = y.copy()
norm_cols_X = (X ** 2).sum(axis=0)
resids = []
prev_mse = 10000
for n_iter in range(maxiter):
for ii in range(p):
beta_ii = beta[ii]
# Get current residual
if beta_ii != 0.:
R += X[:, ii] * beta_ii
tmp = np.dot(X[:, ii], R)
# Soft thresholding
beta[ii] = fsign(tmp) * max(abs(tmp) - alpha, 0) / (.00001 + norm_cols_X[ii])
if beta[ii] != 0.:
R -= X[:, ii] * beta[ii]
mse = np.mean((y - X @ beta)**2)
resids.append(mse)
if prev_mse - mse < tol:
break
else:
prev_mse = mse
return beta, resids

• It seems like you have already answered your question yourself. There's not much complications, since the cost function is convex, and coordinate descend should work without problems. Aug 14 at 20:22
• @SextusEmpiricus great, didn't know if alternating would degrade the convergence. Aug 14 at 20:24