I am using the iterator of
sklearn and i've noticed that i must include a process of feature selection on my experiment. I've seen that it must not consider
testdata for feature selection (for wrapping methods).
Therefore, i couldn't find a good approach to implement using the default
StratifiedKFold loop. Should i perform a feature selection on each fold (
for iteration) and then train my classifier with the reduced features?
Just for clearance, here's my desired experiment:
- Select best features from original dataset
- Balance the classes with SMOTE (must select features first)
- Apply cross-validation
Here is the
for loop of cross validation:
mlp_acc =  adaboost_acc =  # The LOOP - Where do i apply feature selection? for train_index, test_index in skf.split(X, y): clf = MLPClassifier(hidden_layer_sizes=(20),verbose=10, learning_rate_init=0.5, max_iter=2000, activation='logistic', solver='sgd', shuffle=True, random_state=30) adaboost_clf = AdaBoostClassifier(base_estimator=DecisionTreeClassifier(max_depth=10, random_state=50), random_state=40) X_train, X_test = X[train_index], X[test_index] y_train, y_test = y[train_index], y[test_index] # Normalizing data with MinMaxScaler X_train = sc_X.fit_transform(X_train) X_test = sc_X.transform(X_test) X_train, y_train = makeOverSamplesSMOTE(X_train,y_train) adaboost_clf.fit(X_train, y_train) adaboost_pred = adaboost_clf.predict(X_test) clf.fit(X_train, y_train) clf_pred = clf.predict(X_test) # Append accuracies to the accuracy array of each classifier mlp_acc.append(balanced_accuracy_score(y_test, clf_pred)) adaboost_acc.append(balanced_accuracy_score(y_test, adaboost_pred))
I'd be glad for some guidance considering i'm using