5
$\begingroup$

Say I want to predict the cancer rate(regression)/predict the whether a person has cancer or not(classification). The data intrinsically has few cancer patients/low cancer rate, say 1/200. And the data set is good and large enough, say more than 100,000.

Now the question is: should I use certain sampling strategy to balance the data before I apply any regression/classification algorithm?

From my perspective, the reason we need to balance the data is because the data we get doesn't follow the natural distribution, it's bad, like a 10/90 male/female. But right now, we have a good data which follows the natural distribution, should we balance the data?

I'm also wondering if things are different for classification versus regression. Though low cancer rate, is it still okay to do the regression without sampling?

Any high-level/detailed ideas are appreciated:)

$\endgroup$
0
$\begingroup$

There are some very good answers in this thread.

Does an unbalanced sample matter when doing logistic regression?

Also, your setting is a classic setting where you would have high cost if you say someone does not have cancer, but in they fact they do.

$\endgroup$
  • 2
    $\begingroup$ Thanks Tom! Adjusting the penalty is definitely a good advice! But I think it doesn't conflict with the sampling part, right? $\endgroup$ – G. Yu Aug 28 '18 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.