# Improvement on duplicating instances

I have a task of Relationship extraction. There are some set of predefined relations in the corpus. I need to train classifier to recognize the type of relation or the lack of relation between every pair of nouns.

I chose to use a multi-class implementation of SVM. The problem was that I have different number of examples for different relations, for example, for relation type 1 I have 1000 examples in gold standard annotation, however for relation type 2 only 50 examples. The result of classification was very good result on type 1 and very bad on type 2 on validation set.

The solution was to use the same examples from the type 2 several times in my train set, such that the overall number of example of type 1 is close to the number of examples of type 2.

It gives pretty good result on test set. The question is how to explain this, why it actually works. For me it looks counter-intuitive, I didn't expect that using the same examples few times could help.

$$J(\theta) = \sum_i \ell(y_i, f(x_i)).$$
By adding multiple copies of the same instance to your dataset you're essentially telling your optimization procedure you care more about getting that instance wrong than others. Rather than actually adding $k$ copies of an instance $x_i$ to you dataset you could achieve a similar result by multiplying the instance loss for example $x_i$ by $k$.
• I think you're conflating two things, good as in low error and good as in correctly identifies the concept. If $P(Y=2) << P(Y=1)$ then a good then a good (low error) hypothesis may be the classifier $f(x) = 1$ for all $x$. However, if you care about about correctly identifying class $2$, this is not so good. – alto Jan 17 '14 at 20:18