Skip to main content
added 13 characters in body
Source Link
Ben Reiniger
  • 5.1k
  • 1
  • 12
  • 30

This depends at least a little on the model being used. Most often, simple oversampling is asymptotically equivalent to using class weights: an integer weight $w$ on a datapoint has an equivalent effect on loss calculations as duplicating the datapoint $w$ times. Oversampling then is just a discrete version of class-weighting, so asymptotically they should be equivalent, but also for small samples sizes it doesn't seem clear that the discrete version should lead to consistently more or less overfitting.

If your model does any bagging though, things change: by oversampling, you are likely to include a subset of the duplicates of one point, whereas when weighting the subsetting happens before the weights come into play. However, it's still not clear to me that the final effect will be positive or negative in the sense of overfitting. (Unless you're also planning on using out-of-bag scores, in which case this would be quite bad, being very similar to the resampling-before-splitting in cross-validation.)

This depends at least a little on the model being used. Most often, simple oversampling is asymptotically equivalent to using class weights: an integer weight $w$ on a datapoint has an equivalent effect on loss calculations as duplicating the datapoint $w$ times. Oversampling then is just a discrete version of class-weighting, so asymptotically they should be equivalent, but also for small samples sizes it doesn't seem clear that the discrete version should lead to more or less overfitting.

If your model does any bagging though, things change: by oversampling, you are likely to include a subset of the duplicates of one point, whereas when weighting the subsetting happens before the weights come into play. However, it's still not clear to me that the final effect will be positive or negative in the sense of overfitting. (Unless you're also planning on using out-of-bag scores, in which case this would be quite bad, being very similar to the resampling-before-splitting in cross-validation.)

This depends at least a little on the model being used. Most often, simple oversampling is asymptotically equivalent to using class weights: an integer weight $w$ on a datapoint has an equivalent effect on loss calculations as duplicating the datapoint $w$ times. Oversampling then is just a discrete version of class-weighting, so asymptotically they should be equivalent, but also for small samples sizes it doesn't seem clear that the discrete version should lead to consistently more or less overfitting.

If your model does any bagging though, things change: by oversampling, you are likely to include a subset of the duplicates of one point, whereas when weighting the subsetting happens before the weights come into play. However, it's still not clear to me that the final effect will be positive or negative in the sense of overfitting. (Unless you're also planning on using out-of-bag scores, in which case this would be quite bad, being very similar to the resampling-before-splitting in cross-validation.)

grammar
Source Link
Ben Reiniger
  • 5.1k
  • 1
  • 12
  • 30

This depends at least a little on the model being used. Most often, simple oversampling is asymptotically equivalent to using class weights: an integer weight $w$ on a datapoint has an equivalent effect on loss calculations as duplicating the datapoint $w$ times. Oversampling then is just a discrete version of class-weighting, so asymptotically they should be equivalent, but also for small samples sizes it doesn't seem clear that the discrete version should lead to more or less overfitting.

If your model does any bagging though, things change: by oversampling, you are likely to includinginclude a subset of the duplicates of one point, whereas when weighting the subsetting happens before the weights come into play. However, it's still not clear to me that the final effect will be positive or negative in the sense of overfitting. (Unless you're also planning on using out-of-bag scores, in which case this would be quite bad, being very similar to the resampling-before-splitting in cross-validation.)

This depends at least a little on the model being used. Most often, simple oversampling is asymptotically equivalent to using class weights: an integer weight $w$ on a datapoint has an equivalent effect on loss calculations as duplicating the datapoint $w$ times. Oversampling then is just a discrete version of class-weighting, so asymptotically they should be equivalent, but also for small samples sizes it doesn't seem clear that the discrete version should lead to more or less overfitting.

If your model does any bagging though, things change: by oversampling, you are likely to including a subset of the duplicates of one point, whereas when weighting the subsetting happens before the weights come into play. However, it's still not clear to me that the final effect will be positive or negative in the sense of overfitting. (Unless you're also planning on using out-of-bag scores, in which case this would be quite bad, being very similar to the resampling-before-splitting in cross-validation.)

This depends at least a little on the model being used. Most often, simple oversampling is asymptotically equivalent to using class weights: an integer weight $w$ on a datapoint has an equivalent effect on loss calculations as duplicating the datapoint $w$ times. Oversampling then is just a discrete version of class-weighting, so asymptotically they should be equivalent, but also for small samples sizes it doesn't seem clear that the discrete version should lead to more or less overfitting.

If your model does any bagging though, things change: by oversampling, you are likely to include a subset of the duplicates of one point, whereas when weighting the subsetting happens before the weights come into play. However, it's still not clear to me that the final effect will be positive or negative in the sense of overfitting. (Unless you're also planning on using out-of-bag scores, in which case this would be quite bad, being very similar to the resampling-before-splitting in cross-validation.)

Source Link
Ben Reiniger
  • 5.1k
  • 1
  • 12
  • 30

This depends at least a little on the model being used. Most often, simple oversampling is asymptotically equivalent to using class weights: an integer weight $w$ on a datapoint has an equivalent effect on loss calculations as duplicating the datapoint $w$ times. Oversampling then is just a discrete version of class-weighting, so asymptotically they should be equivalent, but also for small samples sizes it doesn't seem clear that the discrete version should lead to more or less overfitting.

If your model does any bagging though, things change: by oversampling, you are likely to including a subset of the duplicates of one point, whereas when weighting the subsetting happens before the weights come into play. However, it's still not clear to me that the final effect will be positive or negative in the sense of overfitting. (Unless you're also planning on using out-of-bag scores, in which case this would be quite bad, being very similar to the resampling-before-splitting in cross-validation.)