0
$\begingroup$

I'm evaluating a classifier to detect objects.

For example:
I have a set1 with 50 pictures with the object to detect.
And set2 with 100 pictures without the object.

I determine the true positves (0 in set2), true negatives (0 in set1), false negatives and false positives for those sets.

If I want to merge set1 with set2 (to have a set with and without objects and real values for TP and TP) I have to decrease the size of set2.

Question:

Can I do this with a simple division? (divide the values of TN, TP etc by 2) or do I have to randomly pick 50 pictures of set2 and use these TN's etc.?

Edit: Example:

Set1: TN = 43 FP = 6 FN = 1 TP = 0
Set2: TN = 0 FP = 13 FN = 0 TP = 87
Set2 divided by 2: TN = 0 FP = 7 FN = 0 TP = 44

Set12: TN = 43 FP = 13 FN = 1 TP = 44

I need an explanation to back this up. thx..

$\endgroup$

1 Answer 1

1
$\begingroup$

For the imbalanced data sets classification, downsampling is mostly used in the case when there is huge difference between the size of two groups, while for your case, maybe upsampling (aka duplicate your set1 to alter the balance) is a better strategy. You can also design your own strategy of resampling (bootstrap for example). Besides, studies have shown that Naive Bayesian classifier and decision tree algorithms are insensitive to the class rebalancing, so the performance of your sample merging may also be dependent on your sampling method. You can try several strategies to determine which one works best among them.

(You can also get more details on this topic from this pdf on imbalanced learning.)

$\endgroup$
4
  • $\begingroup$ I think upsampling is not an option (can't take new pictures and won't use them twice). Down- or undersampling: Seems one way is to choose 50 pictures by random (as mentioned in your linked paper) $\endgroup$
    – ddd
    Nov 21, 2013 at 11:51
  • $\begingroup$ I've updated the question with an example. $\endgroup$
    – ddd
    Nov 21, 2013 at 12:19
  • $\begingroup$ Why down or upsampling is required? Isn't it enough to use the correct prior on the classes? $\endgroup$ Nov 21, 2013 at 12:32
  • $\begingroup$ not sure with the prior.. $\endgroup$
    – ddd
    Nov 22, 2013 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.