Random Forest aims to combine many decision trees to make good predictions for testing data in regression and classification. It is an ensemble learning method.
I have a dataset with 100 samples, each of which has 400 features. I want to classify these samples. When I try to perform random forest classification, I get very low accuracy such as 0.53. According to some resources, there is no need of feature selection when applying random forest because it is very powerful method, and it chooses most important features. However, some people told me that you have many features; hence, at first you have to perform feature selection or pca before random forest classification.
I generally choose 10 maybe 20 features when performing random forest. It has a special parameter which specifies max features, and I choose 20 or 30 decision trees for classification. However, I cannot get a good accuracy.
What do you think about this feature selection issue ? Do we have to perform feature selection or pca before random forest ? Actually, I reduce the number of features during implementing random forest by using max_feature parameter, but it does not work.
rf_clf = RandomForestClassifier(criterion="entropy", n_estimators=20, max_features=10, n_jobs=2) cv_kf = KFold(n_splits=5, shuffle=True, random_state=seed) for train_index, test_index in cv_kf.split(features): train_features = features[train_index] train_labels = labels[train_index] test_features = features[test_index] test_labels = labels[test_index] rf_clf.fit(train_features, train_labels) predicted_labels = rf_clf.predict(test_features) print(accuracy_score(test_labels, predicted_labels))