how to deal with categorical features (with distinct 10000+ values) other than conversion to one-hot encode and ordinal

Machine Learning Problem : I have a doubt in one of my feature which has an categorical value 1. One way of dealing with it would be like converting those values into numbers means in ordinal form. But the problem with this would be my model could have learned the relation as 1<2<3 which is not true in terms of categorical values 2. Other way would be converting values into one hot encoding but it could increase the dementionality and can make my data very sparse So how to deal with that type of problem?

Some of people said to use model like xgboost which handles the categorical values but how they internally tackle that?

• Tell us, please, why this is a prob!em: memory? Use sparse matrices. To many parameters to learn? Maybe some variant of lasso. Other?? And: most importantly: what do you want to do with your data?? – kjetil b halvorsen Sep 7 '18 at 5:07
• Yes memory is a problem and i also want to use it as its an important parameter for judgement – Ajay Sep 7 '18 at 5:37
• Then you can look into sparse matrix representation of your data. Tosay more we really need to know your goal. Is this a regression problem? – kjetil b halvorsen Sep 7 '18 at 5:41
• Its an classification problem – Ajay Sep 7 '18 at 5:42
• How many classes? – kjetil b halvorsen Sep 7 '18 at 5:43

Software: glmnet implements lasso for logistic regression (among other things), and uses sparse matrices out of the box. Try it!