I have several classification systems which I want to compare. The way I evaluate each one of them is using precision, recall and f-score. I want to compare the performance of every two of them using some significance test to find out if the difference in the performance is due to noise or not.
My question is - what would be a good choice for significance test, given the fact that I want to use the f-score values as an input. So far, most of the tests I have seen compare means w.r.t. standard deviation, but I am not sure this is what I need in my case.