This maybe a bit of a strange question, but, I've been wondering about it for a while. Let's say I am training a nonlinear regression model like a neural network on a dataset that has a mix of categorical and numerical variables. The categorical variables have no special ordering. So, let's say one is colour, and my dataset only contains 4 colours. When I train my model, it will only train against the colours in my dataset.

Let's say now, I want to know what the effect on the output variable will be given some 5th colour that was not in the training set. Will this even work? R gives an error when I tried this about new factor levels. Is this even a Machine Learning problem anymore?


Your neural network, or any other machine learning algorithm, can't speak a language you use (say, it's English), so if colours in dataset are "red", "green", "blue" and "yellow", these are the only words to describe colour a network understands. So it doesn't know what to do with "white" or "black".

A bit more precisely: every machine learning algorithm produces formula (possibly highly complicated) that takes values of independent variables and returns prediction of dependent variable. These formulas use only levels of categorical variables that appeared in training sample. Why? Beacuse there is no way to find connection between dependent variable and levels that algorithm have never seen. That is why it can't deal with new levels.

Back to colours: You can code them with RGB or CMYK or any other system. Then your algorithm can deal with new ones. But, colours serve, I suppose, as an example here. It would be difficult to code religion similar way..

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.