What machine-learning algorithm do you use if you have an attribute matrix that was rows = samples and cols = attributes and a target vector that matched the attributes to a specific classifier?

Would I want something like a neural network? Unfortunately, I have many more attributes than samples but only categories to classify (i.e. 0,1, and 2). Considering this, what would be the best algorithm to use for a dataset like the one below? Values will be positive floats.

My data looks something like the DataFrames below. I'm trying to use the attributes to predict the target. I use Python for everything so I was going to try sklearn or tensorflow.

In the end, I would like to have a model that predict the classifier and ALSO to be able to output the important variables that were useful in the model prediction.

import pandas as pd
DF_attributes = pd.DataFrame([
                               [0.1, 12, 0.3, 0.4, 3.4],
                               [0.3, 15, 0.2, 0.7, 6.9],
                               [0.5, 10, 0.8, 0.4, 5.3],
                               [0.6, 13, 0.5, 0.5, 6.3],
                               [0.4, 11, 0.8, 0.4, 7.3],
                               [0.3, 11, 0.4, 0.5, 6.3]

SR_target = pd.Series([0,0,1,0,2,1],index=["s1","s2","s3","s4","s5","s6"])
    att_1  att_2  att_3  att_4  att_5
s1    0.1     12    0.3    0.4    3.4
s2    0.3     15    0.2    0.7    6.9
s3    0.5     10    0.8    0.4    5.3
s4    0.6     13    0.5    0.5    6.3
s5    0.4     11    0.8    0.4    7.3
s6    0.3     11    0.4    0.5    6.3
s1    0
s2    0
s3    1
s4    0
s5    2
s6    1
dtype: int64

closed as too broad by Sycorax, Wayne, Peter Flom Mar 13 '16 at 2:18

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.


I am guessing that your data will be much larger. If you are interested in using Python then for Neural Networks (NN) you can look into the library you mentioned tensorFlow but I also recommend looking into theano and lasange.

There are however many different algorithms that I would suggest for this type of classification based off of personal experience. There is an extension of gradient boosting machines library in both Python and R called xgboost. This particular library is very powerful and is responsible for a lot of winning algorithms on Kaggle.

Finally, I think you will find a fairly good classification in both the RandomForestClassifier and ExtraTressClassifier found in the sklearn.ensemble modules in python. In general, these algorithms perform fairly well and are also robust to things like missing values and outliers.


Just to add a little more to this, another way to create a more powerful classifier would be to run all of the different algorithms mentioned above and take a majority vote of the classification.

  • $\begingroup$ w/ random forests, can you output which attributes were useful in making useful models? When I was doing linear regressions, I would do cross validation and then output the non-zero coefficeints (attributes) to a list that I could look at later. $\endgroup$ – O.rka Mar 13 '16 at 0:29
  • 1
    $\begingroup$ Yes absolutely, one of the great things about these models is the aspect of variable importance. You, can find which variables are important to each model and incorporate the variables that help model your outcome the best! $\endgroup$ – RDizzl3 Mar 13 '16 at 0:35
  • $\begingroup$ Hey thanks the help. Is there any benefit of using a neural network for something like this or are random forests the best way for this type of data? $\endgroup$ – O.rka Mar 13 '16 at 0:36
  • 1
    $\begingroup$ My opinion may be a little biased but I would stick with the random forests or the gradient boosting machine. Reason, being is that NNs can be hard to tune. For a really comprehensive approach I recommend looking at this tutorial (also in python) explaining the mechanics of NNs. iamtrask.github.io/2015/07/12/basic-python-network. This has really helped me when it comes to NNs. $\endgroup$ – RDizzl3 Mar 13 '16 at 0:39

Not the answer you're looking for? Browse other questions tagged or ask your own question.