This maybe a bit of a strange question, but, I've been wondering about it for a while. Let's say I am training a nonlinear regression model like a neural network on a dataset that has a mix of categorical and numerical variables. The categorical variables have no special ordering. So, let's say one is colour, and my dataset only contains 4 colours. When I train my model, it will only train against the colours in my dataset.
Let's say now, I want to know what the effect on the output variable will be given some 5th colour that was not in the training set. Will this even work? R gives an error when I tried this about new factor levels. Is this even a Machine Learning problem anymore?