Is there a way to do Coordinate descent but depending on the variable change the method applied to find the coefficient?
For example, apply a LASSO constraint to a predefined 3 variables and Ridge to the other 5? Or is this just elasticnet with a ratio parameter from 1 (meaning LASSO) to 0 (meaning Ridge) depending on the variable?
Main purpose is to be way more aggressive towards certain variables and shrink them to 0 while keeping others more intact.
I found this code to do LASSO with CD, could I just change the beta calculation in the loop or are there more complications with the CD algo?
def lasso_nb(X, y, alpha, tol=0.00001, maxiter=50000): n, p = X.shape beta = np.zeros(p) R = y.copy() norm_cols_X = (X ** 2).sum(axis=0) resids =  prev_mse = 10000 for n_iter in range(maxiter): for ii in range(p): beta_ii = beta[ii] # Get current residual if beta_ii != 0.: R += X[:, ii] * beta_ii tmp = np.dot(X[:, ii], R) # Soft thresholding beta[ii] = fsign(tmp) * max(abs(tmp) - alpha, 0) / (.00001 + norm_cols_X[ii]) if beta[ii] != 0.: R -= X[:, ii] * beta[ii] mse = np.mean((y - X @ beta)**2) resids.append(mse) if prev_mse - mse < tol: break else: prev_mse = mse return beta, resids