I am trying to classify invoice data into UNSPSC categories. I have 2.2 million invoices and there are around 250 UNSPSC categories the data needs to be divided into. I have labeled data divided into training and test data sets.
The data has the following structure:
UNSPSC-code Product Price Seller 809080 Apple Juice 10 Super Brugsen 802830 iPhone 6000 Electronics Center 902310 Hat 40 Funny Hats Inc
For each variable I have many levels (1000-12000), thus it is not feasible to use random forests for classifications. Random Forests have a limit of around 54 levels in current implementations in R. I also tried support vector machines, but they have no way of handling the memory requirements.
Is there a classification algorithm appropriate for this problem that does not have the same limitations as RF and SVM? I am currently considering doing a KNN search using a custom distance measuring algorithm, but I probably again run into the problem of memory, as KNN will require a distance matrix.