Should I make decisions based on micro-averaged or macro-averaged evaluation measures?

I ran a 10-fold cross validation on different binary classification algorithms, with the same dataset, and received both Micro- and Macro averaged results. It should be mentioned that this was a multi-label classification problem.

In my case, true negatives and true positives are weighted equally. That means correctly predicting true negatives is equally important as correctly predicting true positives.

The micro-averaged measures are lower than the macro averaged ones. Here are the results of a Neural Network and Support Vector Machine: I also ran a percentage-split test on the same dataset with another algorithm. The results were: I would prefer to compare the percentage-split test with the macro-averaged results, but is that fair? I don't believe that the macro-averaged results are biased because true positives and true negatives are weighted equally, but then again, I wonder if this is the same as comparing apples with oranges?

UPDATE

Based on the comments I will show how the micro and macro averages are calculated.

I have 144 labels (the same as features or attributes) that I want to predict. Precision, Recall and F-Measure are calculated for each label.

---------------------------------------------------
LABEL1 | LABEL2 | LABEL3 | LABEL4 | .. | LABEL144
---------------------------------------------------
?   |    ?   |    ?   |   ?    | .. |     ?
---------------------------------------------------

Considering a binary evaluation measure B(tp, tn, fp, fn) that is calculated based on the true positives (tp), true negatives (tn), false positives (fp), and false negatives (fn). The macro and micro averages of a specific measure can be calculated as follows:  Using these formulas we can calculate the micro and macro averages as follows:  So, micro-averaged measures add all the tp, fp and fn (for each label), whereafter a new binary evaluation is made. Macro-averaged measures add all the measures (Precision, Recall, or F-Measure) and divide with the number of labels, which is more like an average.

Now, the question is which one to use?

• I don't know what you mean by "micro" and "macro" and I doubt I'm the only one – shadowtalker Jun 14 '15 at 19:47
• When you ask which to use, what is the intended use? Choosing between the two methods, summarizing results, or something else? – Sean Easter Jun 20 '15 at 16:05
• The intended use is to figure out which model is the most superior, and to tell something about how well it performs. I found out that Micro measurements are superior according to: Forman, George, and Martin Scholz. "Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement." ACM SIGKDD Explorations Newsletter 12.1 (2010): 49-57. – Kenci Jun 29 '15 at 21:30
• @Kenci, I belive you should post that as a reply to your own question and confirm it as being the right anwer. Thanks for the reference! – fnl Jul 27 '15 at 9:58
• – Franck Dernoncourt Dec 28 '16 at 17:08