There is no real answer to your question, because it really depends on what you are trying to archive, i.e. is your goal to get a very high classification accuracy or is it rather data exploration?

If you are purely interested in the classification, you should ask yourself the following questions:

1. Do I expect the same class priors for new samples? If yes, any over or under-sampling will lead to a bad model by definition, since you essentially train the model on a different distribution.

2. What are the consequences of misclassifying a sample? In many cases, the cost of misclassifying a sample is not the sample for all classes, e.g. falsely assign a model to the 'bad document' class might have less sever consequences than assigning it to other classes.

Generally, a model will always try to minimize the loss and it doesn't care how this is archived. In a balanced context, this is solely done by learning correlation between predictors and the response, however in cases of class imbalance, the model will also learn the prior distribution, which is independent of the predictors. This is not a misbehavior of the model in case the actual distribution has these priors! (In this context I want to link a very good answer by Stephan Kolassa about the general issues when evaluating models based on accuracy.)

If you are less interested in the actual classification but more in question such as 'what are the main predictors for the response?', 'do predictors interact?' or 'how big is the deterministic component / the learnability of this problem?', it can make sense to balance classes such that the model doesn't learn the priors but rather the associations between predictors and response, since those could be mask in by class imbalance, especially if you deal with sparse data. However, keep in mind that the resulting model is unfit for classifying data following the original distribution.

There is no real answer to your question, because it really depends on what you are trying to archive, i.e. is your goal to get a very high classification accuracy or is it rather data exploration?

If you are purely interested in the classification, you should ask yourself the following questions:

1. Do I expect the same class priors for new samples? If yes, any over or under-sampling will lead to a bad model by definition, since you essentially train the model on a different distribution.

2. What are the consequences of misclassifying a sample? In many cases, the cost of misclassifying a sample is not the same for all classes, e.g. falsely assign a model to the 'bad document' class might have less sever consequences than assigning it to other classes.

Generally, a model will always try to minimize the loss and it doesn't care how this is archived. In a balanced context, this is solely done by learning correlation between predictors and the response, however in cases of class imbalance, the model will also learn the prior distribution, which is independent of the predictors. This is not a misbehavior of the model in case the actual distribution has these priors! (In this context I want to link a very good answer by Stephan Kolassa about the general issues when evaluating models based on accuracy.)

If you are less interested in the actual classification but more in question such as 'what are the main predictors for the response?', 'do predictors interact?' or 'how big is the deterministic component / the learnability of this problem?', it can make sense to balance classes such that the model doesn't learn the priors but rather the associations between predictors and response, since those could be mask in by class imbalance, especially if you deal with sparse data. However, keep in mind that the resulting model is unfit for classifying data following the original distribution.

There is no real answer to your question, because it really depends on what you are trying to archive, i.e. is your goal to get a very high classification accuracy or is it rather data exploration?

If you are purely interested in the classification, you should ask yourself the following questions:

1. Do I expect the same class priors for new samples? If yes, any over or under-sampling will lead to a bad model by definition, since you essentially train the model on a different distribution.

2. What are the consequences of misclassifying a sample? In many cases, the cost of misclassifying a sample is not the sample for all classes, e.g. falsely assign a model to the 'bad document' class might have less sever consequences than assigning it to other classes.

Generally, a model will always try to minimize the loss and it doesn't care how this is archived. In a balanced context, this is solely done by learning correlation between predictors and the response, however in cases of class imbalance, the model will also learn the prior distribution, which is independent of the predictors. This is not a misbehavior of the model in case the actual distribution has these priors! (In this context I want to link a very good answer by Stephan Kolassa about the general issues when evaluating models based on accuracy.)

If you are less interested in the actual classification but more in question such as 'what are the main predictors for the response?', 'do predictors interact?' or 'how big is the deterministic component / the learnability of this problem?', it can make sense to balance classes such that the model doesn't learn the priors but rather the associations between predictors and response, since those could be mask in by class imbalance, especially if you deal with sparse data. However, keep in mind that the resulting model is unfit for classifying data following the original distribution.

In this case, it might make sense to balance the dataset, but keep in mind that the resulting model is unfit for classifying data following the original distribution.

There is no real answer to your question, because it really depends on what you are trying to archive, i.e. is your goal to get a very high classification accuracy or is it rather data exploration?

If you are purely interested in the classification, you should ask yourself the following questions:

1. Do I expect the same class priors for new samples? If yes, any over or under-sampling will lead to a bad model by definition, since you essentially train the model on a different distribution.

2. What are the consequences of misclassifying a sample? In many cases, the cost of misclassifying a sample is not the sample for all classes, e.g. falsely assign a model to the 'bad document' class might have less sever consequences than assigning it to other classes.

Generally, a model will always try to minimize the loss and it doesn't care how this is archived. In a balanced context, this is solely done by learning correlation between predictors and the response, however in cases of class imbalance, the model will also learn the prior distribution, which is independent of the predictors. This is not a misbehavior of the model in case the actual distribution has these priors! (In this context I want to link a very good answer by Stephan Kolassa about the general issues when evaluating models based on accuracy.)

If you are less interested in the actual classification but more in question such as 'what are the main predictors for the response?', 'do predictors interact?' or 'how big is the deterministic component / the learnability of this problem?', it can make sense to balance classes such that the model doesn't learn the priors but rather the associations between predictors and response, since those could be mask in by class imbalance, especially if you deal with sparse data. However, keep in mind that the resulting model is unfit for classifying data following the original distribution.