1
$\begingroup$

I have a linear model (ElasticNet) which tries to predict customer satisfaction of a supermarket chain. In my data set,I have various features in there like demographics and features of the store, where the customer was interviewed and obv. the grade the customer gave (from 0-100). Lets say, the model could predict the customer satisfaction within an error of +/-1. I now added variables that are depicting lockdowns, so i.e. if the customer was interviewed in a store in Illinois that had a school lockdown at that time, the variable school_lockdown was 1 and if not then 0, etc. I also added the cases of Covid-patients in the relevant state till then. After adding the data and letting the model run again, the model performed much worse and the error of the prediction increased to ~3-4.

It is not important for me why in this particular case, the model performed worse, but I am interested in knowing if there is ever "too much data" or "bad data". I would have guessed, if the additional data is useless, measured wrong or it's just random numbers, that the model would just ignore the variable since it doesnt help making predictions and keep the calculation with the current variables. Am I wrong here?

$\endgroup$

1 Answer 1

2
$\begingroup$

Adding variables that do not have predictive value can worsen the performance of your model. Note that adding more observations will not, given that they are sampled correctly.

As you point out, you would expect that a useless variable would be identified as such and just ignored, and in many cases it will, but not always. Think of this simplified example:

I want to predict a yes/no outcome, I have $k$ variables that actually have predictive value, and $n_{train}$ observations in my training set. I train my model and test it on the test set. Then I want to investigate if flipping a coin will help in my prediction. So, I flip a coin $n_{train}$ times and add the result as a variable. Since the coin flipping is not related to my outcome in any way, there is a fair chance that my model will ignore this new variable, but not necessarily. There is a positive probability that the result of the coin flipping by chance will be related to the outcome in my training set - it can even be a perfect match. However, this relationship will not hold for the test set, and my prediction worsens for the test set.

The more variables you add to your model, the higher the probability of one of them being correlated to the outcome in the training set by chance. In the coin flipping example, this would be flipping a number of coins $n_{train}$ times each, and adding each of them as a separate variable. Eventually, with enough coins, one of them will be a match to the outcome in the training set.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.