When training a multinomial classifier (7 different classes) with different feature sets, I am noticing that the learning curve error always peaks around the number of training samples that are equal to the number of features used in training. I am using k-fold cross validation with k=10 for generating the learning curve.
In the example below, I am using around 500 features for training. I am using a Gaussian Discriminant Analysis model for learning. If I change the number of features, the peak follows.
Is this expected? If so, what is the fundamental reason behind such a behavior?