I am using perceptron to solve a classification problem. I have a limited amount of features (26) and iterate through all possible combinations of them. A combination of two features [feature_a, feature_b] results in better accuracy than the same combination with additional third feature [feature_a, feature_b, feature_c]. Is there any reasonable explanation for that apart from a bug in my code?
I tried replacing perceptron by naive bayes classifiers, the pattern persists.
I am using scikit-learn Python library for perceptron and NB implementations, if that's important.