I am attempting a classification problem where I am trying to assign a row of data to a category that has about 1,000 levels. My predictors are codes that are categorical and can be any of 10,000 possible values. I have between one and ten of these codes for each row of my data. The codes are not independent of each other. The standard practice of converting to indicator variables seems impracticable because of the large number of possible variables. In many ways this analysis would be similar to a spam classifier, except with more than two outcome classes (spam vs. not spam).
One approach to this that worked for me was a Naive Baye's approach. I calculated probabilities of belonging to the class given that these predictors appeared. The assumption of independence among the predictors was definitely violated, so my predicted probabilities did not match reality, but it allowed me to get a classification rate of over 50%. This article was helpful when I wrote the code https://monkeylearn.com/blog/practical-explanation-naive-bayes-classifier/