I am training a tagger to predict whether or not a "word" is a proper noun or not. To do this I take in a list of "words" and their tags for part of speech. I then change all tags that aren't the tag for proper noun to a singular tag. From there I run a method to extract certain features from this data. I train a logistic regression model using train test split.
My model was performing with about a 95.5% accuracy. Then I added a second data set (formatted the same) prior to any manipulation of the input data - essentially just elongating the data I was already using. This dropped my accuracy about 2 percentage points, the complete opposite of what I was expecting. No other code was changed in between the tests.
Are there any common reasons for this sort of issue? I never would have thought adding data (a significant amount) could result in a lower accuracy. Additionally, is there a way to fix this problem? Thank you to anyone who helps!