# Why are all Lasso coefficients in model 0.0?

I'm using from sklearn.linear_model import Lasso in Python 2.7.6

I wrote a script that I've used for doing a Lasso regression for my Features (X) and my Targets (y) . I've used it before and it works, I'm using it on a new dataset (completely different type of data) and I'm getting all 0 coefficients.

What does this mean? Is there anything I can change or tweak to get data?

I've tried different alpha parameters. Here's my function below. I use this class system to store my models and stuff. Let me know if it is confusing or needs to be generalized. I think it's pretty straight forward. My notation is DF_ = DataFrame, D_ = Dictionary, SR_ = Series

#Create the models for store them
from sklearn.cross_validation import LeavePOut
from sklearn.linear_model import Lasso
import time
from collections import defaultdict

class Models:
def __init__(self,target=None,description=None,models=[],duration=0.0):
self.target = target; self.models = models; self.duration = duration; self.description = description
def summation(self):
return(float(sum([q[1] for q in self.models])))
def score(self):
return(self.summation()/len(self.models))

def synthesis(description, query_targets, D_target_Models, DF_attributes, DF_targets,alpha = 1):
"""
Parameters:
[description] key for dictionary of D_target_Models that stores instances of class
[query_targets] list of targets to make models for in DF_targets
[D_target_Models] dictionary of dictionaries:
Outer dict: {description:targets};
Inner dict: {target:model_instance}
[DF_attributes] Pandas DataFrame of attributes (index = sample, column = attribute)
[DF_targets] Pandas DataFrame of targets (index = sample, column = targets)
[alpha] lambda for regression method
"""
lpo = LeavePOut(len(DF_attributes.index)/1000, p=2)
#Check order of indices
if (list(DF_attributes.index) == list(DF_targets.index)) == True:
#         X.index = Y.index = range(len(X.index))
for target in query_targets:
#Create target instance
D_target_Models[description][target] = Models(target=target)
#Get query column for target
SR_target = DF_targets[target]

#Create and train models
models = []
for train_indices,test_indices in lpo:
#Check if all test sets have values
#NOTE!(These conditionsaren't essential for understanding the script.  It's how I ensured there were no NAs)
condition_1 = all([(pd.isnull(SR_target.iloc[test_i]) == False) for test_i in test_indices])
condition_2 = DF_attributes.iloc[test_i].isnull().values.any() == False
condition_3 = None #Impute missing data on DF_attributes
conditions = [condition_1,condition_2]

if all(conditions) == True: #Assumes data is present for all features
#Create model
duration_start = time.time() #So I can time the modeling, not essential
model = Lasso(alpha=alpha)

#Update training indices with non-null target/sensitivity indices
train_indices = [train_i for train_i in train_indices if pd.isnull(SR_target.iloc[train_i]) == False]
#Assign X and y
train_X = DF_attributes.iloc[train_indices,:]
test_X = DF_attributes.iloc[test_indices,:]
train_y = SR_target.iloc[train_indices]
test_y = SR_target.iloc[test_indices]
#Fit model
model.fit(train_X,train_y)

#Predict
predicted_values = model.predict(test_X)
correct_values = test_y
accuracy = int((predicted_values[0] > predicted_values[1]) == (correct_values[0] > correct_values[1]))
if accuracy == 1:
if len(set(model.coef_)) > 1:
print(set(model.coef_)) #ALL COEFFICIENTS ARE 0.0
#Store models
models.append((model,accuracy,test_indices))

#Store time for models
D_target_Models[description][target].models = models
D_target_Models[description][target].duration = float(time.time() - duration_start)
return(D_target_Models)
else:
return("DF_attributes.index != DF_target.index")

• It it possible that your variables simply aren't strongly related to the response? (Note that if you want someone to read through your code & look for problems, that would be off topic here--you could try Code Review.) – gung - Reinstate Monica Oct 28 '15 at 22:26
• Is that what a 0 coefficient would mean for all the attributes? I ran it with LassoCV instead of Lasso and got coefficients. So does that mean that my alpha was the problem? – O.rka Oct 28 '15 at 23:02
• That could have been it. – gung - Reinstate Monica Oct 28 '15 at 23:03

Here, the key fact about LASSO regression is that it minimizes sum of squared error, under the constraint that the sum of absolute values of coefficients is less than some constant $c$. (See here.) So, for all of the coefficients to be zero, there must be no vector of coefficients with summed absolute value less than $c$ that improves error.
$$\sum_{i = 1}^n (Y_i - X_i^T\beta) + \lambda\sum_{j=1}^p|\beta_j|$$
As put in the tutorial referenced above, "If $\lambda$ is sufficiently large, some of the coefficients are driven to zero, leading to a sparse model." For it to be the case that zero coefficients minimize this function, $\lambda$ must be large enough that any improvement in error (the left term) is less than the added loss from the increased norm (the right term).
It's common to use cross validation to set this parameter such that the model minimizes CV error. This could be why LassoCV gave you different results—it may have set $\lambda$ for you.