Most classification algorithms will only perform optimally when the number of samples in each class is roughly the same. One way of addressing the issue of skewed datasets where the minority class is outnumbered by one or more classes is to re-sample the dataset in order to arrive at a more robust and accurate decision boundary. Re-sampling techniques can be divided broadly into four categories: undersampling the majority class, oversampling the minority class, combining over and under sampling, and creating an ensemble of balanced datasets. See Learning from Imbalanced Data by He and Garcia for an extensive review.
Common undersampling strategies include random undersampling (removing points uniformly at random), cluster centroids (compressing majority class by K-means centroids where K is determined based on the level of undersampling), Tomek links (removing overlaps between classes) and others. Common oversampling strategies include random oversampling, SMOTE (generating synthetic examples using KNN), and ADASYN (weighted distribution for minority class examples according to their level of difficulty in learning).
The above methods and more are implemented in the imbalanced-learn library in Python that interfaces with scikit-learn. I recommend trying an combined method such as SMOTE + Tomek links to see if classification accuracy improves on a balanced dataset.