Ok, I have a classification problem in which i need to classify instances into one of 5 classes.
The class distribution of the classes is imbalanced however. These are the frequencies of instances for each class:
class 1: 129
class 2: 2
class 3: 187
class 4: 18
class 5: 285
So this is a multi-class classification problem. With imbalanced class distributions like this, I read papers that recommend to use cost sensitive classifiers. Most literature that I read however has pretty much set out how to do this with binary classification problems but not for multi-class problems. One important aspect, making a cost matrix, is relatively easy when you have a binary class problem. For example if you have 10 A and 90 B instances, you can make a cost matrix that penalizes classifying A as B with 9, while penalizing classifying B as A only with 1:
0 1
9 0
So I would like to be able to make a binary classifier problem of my multiclass problem.
For this I use the following things in weka:
First of all I use the metaclassifier: MultiClassClassifier.
In this MultiClassClassifier I put a MetaCost classifier, which on its turn uses for example a J48 classifier. My problem is, that I need to define a cost matrix in that MetaCost classifier. I want that cost matrix to be build differently for each round that the MultiClassClassifier makes.
Is there any way to do this?
