7
$\begingroup$

I am using AdaBoost Classifier to predict values I have. How can evaluate the accuracy of prediction model (I'd like to see how the accuracy of predicted values).

You can check an example here: http://scikit-learn.org/stable/modules/ensemble.html#usage

I found two options : using confusion matrix

from sklearn.metrics import confusion_matrix
cm = confusion_matrix(expected, y_1)

or using cross val score

scores = cross_val_score(clf_1, X_train, y_train)
print scores.mean()

There is also: AdaBoostClassifier.staged_score(X, y) AdaBoostClassifier.score(X, y)

So, I am little bit confused.

One last question: Should I use predict() or predict_proba().

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

In order to get the accuracy of the predication you can do:

print accuracy_score(expected, y_1)

If you want a few metrics, such as, precision, recall, f1-score you can get a classification report:

print classification_report(expected, y_1)

A confusion matrix will tell how many of the samples that were classified are classified according to which label. This will tell you if your classifier confuses some categories.

The functions to get these metrics are independent of the classification model you are using. (So you can easily test an SVM for example)

You should use predict() since this will give the labels of the classified samples. predict_proba will give the propability of a sample belonging to a category

I recommend reading a few of the documentation pages:

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thanks ! Another question regarding GLM Model, when I use the same metrics, I got an error: "Can't handle mix of binary and continuous". Is it different from model to another ? $\endgroup$
    – user3378649
    Commented Apr 25, 2014 at 9:59
  • $\begingroup$ My guess is you are using a regression model and not a classification model. If you want a linear classifier look at SGD, or a linear SVM or perceptron. Have a look at this diagram: peekaboo-vision.blogspot.be/2013/01/… to choose a good model. $\endgroup$
    – Olivier_s_j
    Commented Apr 25, 2014 at 10:04
  • $\begingroup$ I am trying to do a benchmarking between two models for my school project. regression model(GLM), and another classification Algorithm (AdaBoost). In order to discuss the both algorithms I need to compare the prediction accuracy (check GLM example: stackoverflow.com/questions/23215010/problems-using-glm ) $\endgroup$
    – user3378649
    Commented Apr 25, 2014 at 10:24
  • $\begingroup$ I think you are confusing regression and classification. With regression the output takes on a continues value and with classification it will take on a class label. These 2 type of problems have different metrics. Since one is dealing with continues data and the other with discrete data. For example you can't use a confusion matrix when doing regression. A metrics for regression is for example mean squared error $\endgroup$
    – Olivier_s_j
    Commented Apr 25, 2014 at 10:36
  • $\begingroup$ Great Explanation. In this case, how can I measure the accuracy in regression models (where we have continues data). Should we discuss Residuals, variance, pearsona-chi .. (Chech this example: blog.yhathq.com/posts/r-lm-summary.html) $\endgroup$
    – user3378649
    Commented Apr 25, 2014 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.