0
$\begingroup$

I am currently training a model to predict a binary attribute. The model gives the output in range [0, 1]. The metric is TPR@FPR, e.g. I need to achieve maximum True Positive Rate at 0.1 False Positive Rate.

The problem I have is that the model behaves differently across different datasets. For dataset A it gives predictions in [0.1, 0.4] range, but for dataset B it gives predictions in [0.3, 0.8] range. While my model achieves good performance on each dataset separately, when combined the performance decreases significantly, since all points in dataset A are below calculated threshold.

How can I fix this problem?

Some info about the data: this is a CV problem, so all data are images, all images are pre-processed the same way. The dataset differ in both time and location.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.