How LightGBM deal with a new categorical value in the test set Suppose I have the training data set $(X, y)$ where $X$ is my feature space $(x_1, \dots, x_n)$. Let $x_1$ be a categorical feature column. In my test set, if the feature $x_1$ takes a new categorical value that does not exist in my training data set (i.e. one row in $x_1$ has a categorical value that was not in the train data set). How does LightGBM handle it?
 A: LightGBM will not handle a new categorical value very elegantly. The level of elegance will depend a bit on the way that the feature is encoded to begin with. (For that matter most automatic methods of handling categorical variables will also fail.)
More details:
Formally "categorical features must be encoded as non-negative integers". If the new feature value for feature x1 is encoded as a 5 and in training set we had features {1, .., 4} then we are "almost OK". We will have some minor performance degradation in our test set but our predictions for sample instances with categories {1, .., 4} are going to fine; feature values 5 will suffer somewhat as they will treated as feature value 4. Unfortunately in the same scenario if our new feature value for feature x1 is encoded as a 2 most of our  existing feature x1 categories will be messed up because they will be shifted. Now how/why can this later point happen? Python's pandas as well as R's factor variables usually (i.e. by default) create categories in lexicographic order. If therefore we had x1 features values {a,c,d,c,d,a} those would be used as: {0,1,2,1,2,0} but if our test set was like {a,b,c,d} those would be translated as {0,1,2,3}, i.e. what we thought as being d in our training set now is c in our test set.
Work-around: Use target encoding. Simplify things a bit, we create a numeric feature x1_num with the (regularised) mean response variable y per category of the categorical variable x1; we then use x1_num instead of x1. There are quite a few ways of performing that regularised mean estimation (e.g. see the Python package category_encoders for more than a half-a-dozen of them). After doing this target encoding step unseen/new x1 values are assigned our prior estimate value for the feature x1_num. This prior estimate might be a dynamically computed variable (e.g. the mean or the median of our response variable y (or maybe a distinct combination of other explanatory variables - check out for example the dirty_cat which handles typos, etc. in a super smart way)) or simply a value we set manually.
