# What are some good canned classifiers for high-dimensional data with probablistic labels, besides neural nets?

I've got a classification problem where my labels are $N\times4$ matrices of probabilities of class membership, and I've got about 1800 covariates. The covariates are mostly granular, in the sense that an additively separable model (like a penalized multinomial logit) probably would work that well.

I also haven't had much luck with using standard multinomial classifiers, after assigning each observation to the highest-probability class -- xgboost and random forest don't find much.

A neural net is the obvious solution, but tuning the hyperparameters is a huge chore that I'd like to avoid if possible. Are there other good options that require less futzing time?

For background, the class probabilities are all weights from a finite mixture model. I trained it and it fits really well, but I'm only just now realizing that I won't know the class weights for new data unless I know their outcomes! D'oh!

• Support vector machines, nearest neighbors, naïve Bayes, Gaussian processes..
– Sycorax
Commented Sep 3, 2018 at 0:14
• It's worth noting that both xgboost and random forest also require parameter tuning.
– Sycorax
Commented Sep 3, 2018 at 0:23
• @Sycorax nearest neighbors in 1800 dimensions? LOL Commented Sep 3, 2018 at 0:25
• A couple questions. 1) Seems like known structure would be relevant to classifier choice, but I didn't understand the statement about granular covariates and additively separable models. Could you expand on that? 2) What did you fit the mixture model to? Was it data in the same 1800-dimensional space as your regressors, or something else? Commented Sep 3, 2018 at 2:20
• @user20160 Granular in the sense that pixels in an image wouldn't form a good additively separable classifier -- it is aggregations of them that create the image. My data isn't image data (it's a regression problem), but it's the same idea. The mixture model actually is a mixture of 4 neural networks with 1800 inputs. I did this after finding evidence that I wasn't capturing subgroup heterogeneity. I don't want to do it again! But I might have to. Commented Sep 3, 2018 at 11:41

If your problem is related to image processing then you may want to look into the rFerns package, the rFerns function is a mix of random forests and naive bayes. It's extremely fast, efficient, and simple to use, plus it was specifically designed for image processing / object recognition.