0
$\begingroup$

What machine learning algorithm would be the most optimal to use for a classification problem with a small number of classes? In my case, only two. The sample size is also rather small (<100), but the number of predictors is huge (~10 000). All the predictors are continuous floats and (practically) independent of each other. I tried random forest, but this isn't very accurate with continuous predictors. The only algorithm I can think of is multiple linear regression. Are there any algorithms that would be more optimal in this case? By optimal I mean high sensitivity and specificity. Computational performance is secondary. Would also be nice if it were already implemented in an existing Python or R library.

$\endgroup$
0
$\begingroup$

If you try to fit 100 data points with 10000 parameters all at once you will be in deep trouble since you have more variables than constraints! You might want to try doing a first pass where you loop through all of your 10000 variables in a 1 parameters model and seeing which variables, considered individual in a one parameter model, give you the best correlation. From that, you could take the top 3 or so and see if that improves your result. This will fall apart if you data is highly correlated but may give you some insight into which parameters to keep and which to discard.

$\endgroup$
  • $\begingroup$ That's a good suggestion! I do have to trim down the set of classifiers. But I'm still not sure what method I should use. The samples in my case aren't exactly data points but individual patients. I need to be able to classify patients into either a normal(0) or diseased(1) category which are two discreet categories. I tried linear regression, but there's the problem of cutoff - from which value I should consider the patient to be in the 1 (diseased) category. $\endgroup$ – Karl Märka Apr 15 '16 at 23:38
  • $\begingroup$ Yes, the million dollar question that I don't think you can answer without an indepth understanding of your particular problem. My strategy is to start simple and go more complicated as needed. Thus, I would start with just a simple one parameter logistic regression model and score by minimizing the number of false positive and false negative values. You can then identify some likely predictors and then try something more complicated like a decision tree or PLS algorithm. $\endgroup$ – Greg Petersen Apr 18 '16 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.