9
$\begingroup$

Here is my Code for feature selection method in Python:

from sklearn.svm import LinearSVC
from sklearn.datasets import load_iris
iris = load_iris()
X, y = iris.data, iris.target
X.shape
(150, 4)
X_new = LinearSVC(C=0.01, penalty="l1", dual=False).fit_transform(X, y)
X_new.shape
(150, 3)

But after getting new X(dependent variable - X_new), How do i know which variables are removed and which variables are considered in this new updated variable ? (which one removed or which three are present in data.)

Reason of getting this identification is to apply the same filtering on new test data.

$\endgroup$
5
$\begingroup$

There are two things that you can do:

  • Check coef_ param and detect which column was ignored
  • Use the same model for input data transformation using method transform

Small modifications for your example

>>> from sklearn.svm import LinearSVC
>>> from sklearn.datasets import load_iris
>>> from sklearn.cross_validation import train_test_split
>>>
>>> iris = load_iris()
>>> x_train, x_test, y_train, y_test = train_test_split(
...     iris.data, iris.target, train_size=0.7
... )
>>>
>>> svc = LinearSVC(C=0.01, penalty="l1", dual=False)
>>>
>>> X_train_new = svc.fit_transform(x_train, y_train)
>>> print(X_train_new.shape)
(105, 3)
>>>
>>> X_test_new = svc.transform(x_test)
>>> print(X_test_new.shape)
(45, 3)
>>>
>>> print(svc.coef_)
[[ 0.          0.10895557 -0.20603044  0.        ]
 [-0.00514987 -0.05676593  0.          0.        ]
 [ 0.         -0.09839843  0.02111212  0.        ]]

As you see method transform do all job for you. And also from coef_ matrix you can see that last column just a zero vector, so you model ignore last column from data

$\endgroup$
  • $\begingroup$ Hi, How can I identify the Column names of X_train_new. Is there any function? $\endgroup$ – Vignesh Prajapati Aug 27 '15 at 22:08
  • 1
    $\begingroup$ They are in the same order as in the input data set. iris.feature_names $\endgroup$ – itdxer Aug 28 '15 at 9:15
  • $\begingroup$ Yes. Its. I am confused here. It is in the same order. But how do I get their names because some of the columns have been ignored. So, I am not able to get those specific columns which got selected while this process. Can you please help me on this!. $\endgroup$ – Vignesh Prajapati Sep 2 '15 at 8:43
  • $\begingroup$ Did you check the method feature_names in iris variable? It works fine for me. $\endgroup$ – itdxer Sep 2 '15 at 10:48
12
$\begingroup$

Alternatively, if you use SelectFromModel for feature selection after fitting your SVC, you can use the instance method get_support. This returns a boolean array mapping the selection of each feature. Next join this with an original feature names array, and then filter on the boolean statuses to produce the set of relevant selected features' names.

Hope this helps future readers who also struggled to find the best way to get relevant feature names after feature selection.

Example:

lsvc = LinearSVC(C=0.01, penalty="l1", dual=False,max_iter=2000).fit(X, y) 
model = sk.SelectFromModel(lsvc, prefit=True) 
X_new = model.transform(X) 
print(X_new.shape) 
print(model.get_support()) 
$\endgroup$
  • 5
    $\begingroup$ This should be accepted $\endgroup$ – user0 Dec 14 '16 at 23:11
5
$\begingroup$

Based on @chinnychinchin solution, I usually do:

lsvc = LinearSVC(C=0.01, penalty="l1", dual=False,max_iter=2000).fit(X, y) 
model = sk.SelectFromModel(lsvc, prefit=True) 
X_new = model.transform(X) 
print(X.columns[model.get_support()]) 

which returns something like:

Index([u'feature1', u'feature2', u'feature',
  u'feature4'],
  dtype='object')
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.