1
$\begingroup$

I'm using MATLAB R2016a for binary classification (time series prediction) of a financial case. I have a good total accuracy (70~75%) but specificity is about 90% and sensitivity is about 60% and vice versa. Currently I'm using grid search to optimize my classification model (SVM, neural network, etc.) based on total accuracy. My data-set has balanced samples of binary output.

How can I improve results in this case? Can I use any other performance metric to take into account unbalanced sensitivity and specificity (as mentioned, in some cases I have higher specificity than sensitivity and in some cases vice versa.)?

Improve this question
$\endgroup$
1
  • $\begingroup$ I'm not sure how to do this in Matlab, but in R I favor maximizing Kappa over Accuracy, especially when class frequencies are unbalanced. $\endgroup$
    – HEITZ
    May 18, 2016 at 23:36

1 Answer 1

Reset to default
1
$\begingroup$

It seems like you may have answered your own question. You optimized your methods for proportion classified correctly, and so that's what came out nicely, possibly at the expense of other metrics. If you want, e.g., to maximize accuracy under the constraint that that both specificity and sensitivity are at least 65%, then try tuning the models and setting thresholds accordingly. If you care about false positives and misses equally, and the base rate is 50-50 too, then maximizing accuracy, as you've already done, seems like the way to go. But if one is worse than the other, try defining an appropriate loss function and try to maximize that instead.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.