Best machine learning algorithm for loans dataset? I have a dataset of about 75K samples with about 20 features per sample (12 of which are probably important) describing various credit profiles - credit score, late payments, income, etc. Some of the input variables are continuous and some are categorical. The output variable is continuous - the ROI associated with each loan. What is the best supervised learning algorithm to predict the ROI? How do I select the features and weigh them?
I was looking at some examples of SVM but I need something that is not binary (can actually predict the ROI)
 A: If your categorical data are ordered, and therefore could be transformed to continuous (but integer) variables, then the Random Forest algorithm is likely to be a competitive solution which requires very little tuning compared with other methods (such as neural networks). RF will effectively do feature selection (of sorts) internally so you shouldn't need to do any pre-selection before sending the data to the algorithm.
In the case that you have unordered categorical variables then you can create a set of new binary variables representing each categorical one. For instance the variable Animal which could be one of Cat, Dog or Horse would become three variables, Cat, Dog and Horse with one set to 1 and the rest 0 depending on the value of the original variable. There is a succinct technical name for this kind of approach, but I can't remember what that is. It doesn't matter that this might produce lots of new variables as RF scales well with the number of inputs.
A: The question, as asked, is impossible to answer correctly.  It depends on the structure inside your data.  Is there a distance measure on your data such that the average of the closest n-points almost always give a good estimate of the values?  Is there a global structure to the data such as one that would be reflected in a linear or more complex model of the same type?  Or are there a number of independent but possibly overlapping clusters in the data, each with their own structure?  If each of your 20 features can realistically be considered independently for their effect, then learning an additive model might be perfect, but if certain combinations of features change the picture entirely then you need a different kind of learning algorithm.  In order to determine the best supervised learning algorithm for your data you need someone who is skilled in using the different algorithms and is familiar with their characteristics to do some exploratory analysis on your data.    BUT....     An adequate learning algorithm is a completely different question.  Sight unseen, I can tell you that a suitably sized back-propagation neural network, without the non-linearity on the output of the top neuron, will probably do you just fine unless you are extraordinarily unfortunate and have a deceptive collection of data where an inferior solution has a wide basin of attraction.  I can also tell you that random forests are robust and tolerant of many different kinds of structure, though not necessarily the most efficient approach.  
