# Why does this Neural Network have an accuracy drop-off on higher feature sizes

I am currently training different models (neural network, support vector machine, and XGBoost) to predict concentrations of antibiotics required to prevent growth of a bacteria from whole genome sequences.

The features for all models are the number of appearances of all substrings of length 11 (e.g. 'AGTCCGATACG' might appear 23 times in a specific sample). There are roughly 2.1 million features for 2260 samples. Using sklearn's SelectKBest() the feature size is reduced to between 100-10000 features. There are 6 possible classes (1,2,4,8,16,32) which are encoded to 0-5 and most of the data falls under the first and last classes.

Here is what the accuracy looks like against the number of features for the 3 models for a 5 fold cross validation.

The neural network has accuracy's consistent with the other 2 models until about 1700 features, after which the accuracy of the model drops off. I am hoping that I am providing enough information that someone can help answer why this is happening.

Here is what my NN looks like:

    patience = 16
early_stop = EarlyStopping(monitor='loss', patience=patience, verbose=1, min_delta=0.005, mode='auto')
reduce_LR = ReduceLROnPlateau(monitor='loss', factor= 0.1, patience=(patience/2), verbose = 1, min_delta=0.005,mode = 'auto', cooldown=0, min_lr=0)

model = Sequential()