# Unbalanced test data matter?

I have balanced training data: 300 positives and 300 negatives, but the test data is unbalanced: I have 15 positives and 60 negatives. Will the unbalanced test data impact classification accuracy?

Actually, I tried to collect balanced test data (30 pos & 30 neg). The precision on positives are better. Is the reason the balance of test data? Thanks.

## 1 Answer

Depends entirely on what your model's properties and the performance measure are.

Lets take a simple example: suppose your model predicts all positives and half of the negatives correctly on your training set (e.g. accuracy 75%). Assuming the positives and negatives in the test set have similar distributions to those in the training set, the model will probably perform similarly on the test set. In that case the accuracy would drop to $(15+30)/75 = 60\%$.

You have to answer two questions before looking at performance numbers:

1. What properties of the model matter to you?
2. What score measure do you use to capture (1)?

You can eliminate the effect of unbalanced data by using score measures such as area under the ROC curve.