# Are more features always better?

Was discussing with a friend: suppose we have one model that uses 1,000 features and another that uses 100,000 features. Assuming their first 1,000 features are the same, shouldn't the one with 100,000 features always do at least as well as the 1,000 feature model?

I say this because, if there's a correlation between an additional feature and the target variable, it can learn this. If there's no correlation, the model should learn to ignore. So more features should always be at least as good as a model with only a subset of the same features.

My friend claims features can actively hamper model performance so that more isn't always better...how is this possible?

Thanks!

But, say that your friend has the first 100 features and you have 1000 and you're trying to predict housing prices. It may be that location is the 101$^{st}$ feature. There is something to be said for restricting the variance of the model while keeping all of the features---e.g., with shrinkage, dropout, or ensembles.