Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite

I have several confusion tables calculated, see, e.g., table 2 and table 1, but typically I'll have 10 or so. These confusion tables are the result of different classification processes (in fact they are the result of different sensor locations). The row and column labels are the same in each table, only the values and thus the accuracy (the last column) differs.

Now I want to run a test which can tell me which table performs better (i.e, which sensor location is best). I thought of just taking the average of the accuracy, but I would also like to know which rows attribute the most to the difference.

share|improve this question
1  
Perhaps you can clarify a bit more. Are you interested in the type of errors? Are the frequencies in the various categories equal. What do you mean by "which rows attribute the most to the difference"? Does this mean you are interested in testing the difference between rows? Thanks. – Joel W. Aug 2 '12 at 23:39

1 Answer

up vote 1 down vote

I am not exactly sure what you mean by "contribute most to the difference". If you just mean, which have the most errors, you have that in the table: Just subtract the number correct from the row total; or, if it better suits your needs you can do it in terms of proportion that are correct in each row.

Taking overall accuracy assumes that all errors are equally bad. Without context, it's not possible to know if this is reasonable. But, if you can quantify the 'badness' of each type of mistake, you can pretty easily create an index of that.

share|improve this answer
Not all errors are equally bad. The table is the result of classification on a ordinal scale. So the further from the prediction is from the actual class, the worse the error is. – jorrebor Aug 10 '12 at 11:26
OK, so, you can, for example, use the distance between the prediction and the actual as an index of "badness" and then calculate the badness for each table. This tacitly says that the ordinal differences are equal. If you can come up with better values, then you can use those – Peter Flom Aug 10 '12 at 14:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.