Given a data set where each individual data point can be assigned to more than 1 class (a multi-class, multi-label data set), are there any guidelines for calculating oversampling weights, i.e., the probability with which you sample a data point based on the frequencies of the labels within the data set?

This is in the context of multi-label classification; I have a very imbalanced data set.

An obvious answer would be to calculate the weight for each label as the inverse frequency (i.e. 1 / total_number_of_label_appearances), then average up the weights for a given data point; though I'm unsure if there's any better approaches.


Calculating the weight for each label as the inverse frequency, then average up the weights for a given data point is done like so with pandas in Python:

from itertools import chain
from collections import Counter
import pandas as pd

def oversample(df, len_mult=2, random_state=0) -> pd.DataFrame:
    value_counts = Counter(chain(*df[label_col].dropna()))
    weights = 1 / df[label_col].map(
        lambda li: sum(map(value_counts.get, li)) / len(li), na_action='ignore'
    # Fill in average weight for rows w/o labels
    weights.fillna(weights.sum() / weights.count(), inplace=True)
    extra_df = df.sample(
        len(df) * (len_mult-1), replace=True, weights=weights, random_state=random_state
    df = pd.concat([df, extra_df])
    return df

df = pd.DataFrame([['x1', [3,4,5]], ['x2', [0,3]]], columns=['x','y'])

If you are using a probabilistic classifier, that outputs a probability of class membership for each class, then averaging the weights or choosing the weights to minimise the maximum imbalance both seem reasonable. However you would likely to end up with a classifier that grossly over-predicts some classes, so I would subsequently post-process the outputs of the trained model on a label-by-label basis to account for the difference in relative class frequencies in training set and under operational conditions (which basically requires multiplying by the ratio of operational and training set prior probabilities and then re-normalising the probabilities to sum to one). Unlike the re-sampling of the training set, that can be done on a label-by-label basis.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.