I took a method to determine the best features for a classifier from somewhere and I wanted to ask if it is reliable or not:
I was trying to, very simply, select the best features for a Logistic Regression classifier.
Using Lasso().coef_ it turned out that everytime I switched the training and testing data indexs (with a random ShuffleSplit, so in a CrossValidation process), the 0 lasso coefs changed. So, to visualize it:
Loop1 random_seed=1 Lasso_coefs_obtained = [0, 0.9, 0, 0, 0.1] Loop2 random_seed=2 Lasso_coefs_obtained = [0.3, 0, 0, 0.7, 0 ] Loop3 random_seed=3 Lasso_coefs_obtained = [0.1, 0.8, 0, 0.2, 0.2]
In order to determine if a feature coefficient was really null, I would sum the three coefficients lists and determine the 0 elements (which must be those elements that are 0 in the three lists).
Sum_coefs = [0.4, 1.7, 0, 0.9, 0.3]
In my case, I did this method with 10 loops (I have 10 coefficients lists over 72 variables). After 10 loops, only 55 coefficients remain non-zero. So, I delated 17 features for the classifier.
Is this method reliable or I'm just making up an algorithm with no guarantees?