Overfitting due to a unique identifier among features In many beginner ML lectures / tutorials, it's advised to remove those features that uniquely identify the example. For example, if predicting user behavior, a numeric user_id column should be removed. The stated reason is that a powerful classifier would use that column to fit perfectly on the training set, ignoring all the other columns, resulting in a useless model (of course the CV will show that).
I don't understand why a classifier might focus so much on user_id at the expense of the other columns. Any continuous column often has the power to uniquely identify the example. For example, if we use user_distance_from_nearest_store with high enough precision, its likely no two users have exactly the same distance, so the distance may uniquely identify users.
So I can't see why including user_id would cause any more problems that including any other completely unimportant continuous feature. Am I missing something?
(Of course, I understand user_id shouldn't be included, since it can't help the model and can hurt it due to noise. I'm just trying to understand why it might be so influential if it's included by accident.)
Edit: to clarify, by "numeric" column, I meant not categorical. Also, I was talking about a classifier, but the intuition will carry over to a regression.
 A: "Certainly, any continuous column (with enough resolution) has the power to uniquely identify the example." - that's not true for a predictor that is treated as continuous. For instance, if you generate Y and X at random using  uniform distribution, then all of the values of X will most likely be distinct. However, when you regress Y on X using simple linear regression, the fit (in terms of $R^2$) will not be perfect. That's because there is a restriction on relationship between Y and X in the form $Y = x'\beta + \epsilon$. There are only two parameters in the model.
If you treat X as a categorical predictor with N levels where N is the sample size, then there is no such restriction: for each level of X, you are allowed estimate a distinct parameter that describes the average response at that level of X. That is, you end up with N observations and N parameters. For each such parameter, its estimate will be equal to the observed value of Y, so you will end up with a perfect fit. 
One implication is that if your subject ID is formatted as a number and the ML package will see it as a quantitative predictor, then you won't have a problem with it, apart from including a meaningless predictor in your model.
A: The predictive value of an ID field will vary considerable from dataset to dataset, so in some cases it's probably ok to leave in, and in others not. One case where it could have high predictive value (and should be taken out) is where you're trying to predict the age of something, and the ID is assigned according to age (e.g., new accounts or users' IDs are incremented). Other than cases like this, a continuous feature wouldn't provide much value. 
An overfit tree, which makes so many splits that each terminal node has only one observation, is just as likely to overfit on an ID column as any other column. Playing devil's advocate, one might even argue for leaving it in so as to have some sort of benchmark in your dataset for identifying variables that have no value. 
A: Another way to look at it is from the POV of a neural network. It is much easier to memorize which classes belong to which target, than to learn meaningful features. 
Including the identifier also encourages co-adaptation, which overfits on the training set. 
