What machine learning algorithm would be the most optimal to use for a classification problem with a small number of classes? In my case, only two. The sample size is also rather small (<100), but the number of predictors is huge (~10 000). All the predictors are continuous floats and (practically) independent of each other. I tried random forest, but this isn't very accurate with continuous predictors. The only algorithm I can think of is multiple linear regression. Are there any algorithms that would be more optimal in this case? By optimal I mean high sensitivity and specificity. Computational performance is secondary. Would also be nice if it were already implemented in an existing Python or R library.
If you try to fit 100 data points with 10000 parameters all at once you will be in deep trouble since you have more variables than constraints! You might want to try doing a first pass where you loop through all of your 10000 variables in a 1 parameters model and seeing which variables, considered individual in a one parameter model, give you the best correlation. From that, you could take the top 3 or so and see if that improves your result. This will fall apart if you data is highly correlated but may give you some insight into which parameters to keep and which to discard.