How to test the robustness and the performance of a novel classification algorithm? Suppose you have a new algorithm that you want to publish. Are there any best practices and methodologies you usually consider in order to test the robustness and performance of a new method?
The case is a novel classification algorithm of matrices composed by observations x features (e.g. a gene expression matrix).
 A: Test it on several known datasets and use them as benchmarks. Such as the iris dataset http://en.wikipedia.org/wiki/Iris_flower_data_set or whatever is the common dataset in that field.
You would want to compare it in both speed and accuracy.
A: Outlier detection and robustness are two ends of the same stick. 
In robustness your objective is to fit a model on contaminated data such that you find a fit as close as possible as the one you would have had without the outliers. 
Outlier detection tries to find all the outliers that matter in the sample. That is, all points that exert a disproportional pull on the fitted parameter of the model. 
If your algorithm is equivariant in some sense (the exact sense depends on the estimation problem, for example for regression a natural group of equivariance is regression equivariance, for classification, a natural group of equivariance for the vector of fitted probabilities is affine equivariance), then, in many cases the worst possible (or adversary) configuration of outliers, in the sense of being those that are hardest to find, are either known or can be worked out with reasonable confidence. 
Then, you should simulate from and test your procedure preliminary on those (adversary) cases. The bias (or difference between the fit you have on contaminated data and the fit you would have had without the outliers) computed on adversary configuration of outliers is how you want to measure the performance of any anomaly detection procedure.
