0
$\begingroup$

Training machine learning models on an unbalanced dataset: about 3% positive labels, and 97% negative. The modeling goal is to get as many examples as possible with 60% precision on a holdout test set per model predictions. (all numbers are hypothetical.)

I've got some general questions on difference between the two approaches:

  • A) Train and test ONE model on the entire dataset, and
  • B) Train and test MULTIPLE segmentation models on disjoint subsets of the same dataset. These subsets may be based on values of a certain feature, or multiple features.

Q1: Are Approaches A and B essentially the same? I found a segmentation model (trained and tested on a subset of the data based on a certain feature values) that can reach comparable performance to Approach A, per above-mentioned modeling goal, i.e. with 60% precision, the segmentation model has almost as many examples, while the subset is less than 5% of the train set of Approach A. (That is why I started wondering about building more segmentation models on disjoint subsets.)

Q2: If Approach A can not satisfy the modeling goal, what are some sensible next steps, e.g. improve data unbalancedness? improving data quality? try different types of model?

Thanks!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.