I am currently doing classification for some data over 49 classes. After training the model i test its classification accuracy on a data-set consisting of 40 observations for each class.
To put this in context, each class refers to a location on a 2D plane, and these locations are closest to their neighbouring points, also between every 2 classes the distance is equal. So for example class 3 is closest and equally distant to classes 2 and 4.
Now, lets consider Class 1, it has 35 correctly classified observation, but 5 are misclassified, and If we look closer at the misclassified observations, we see that they were classified as 1 observation in class 9 and the other 4 in class 40.
the 1 observation classified in class 9 is not a major issue, however the ones classified in class 40 which is very far away from the true class are very bad for my case.
So is there a way to edit the training process with the aim of achieving less misclassification in far away classes ? even if it means that we would have a higher misclassificaiton as long as they reside in the neighbouring classes.
Just as a note, I am using LDA and SVM, but am not restricted to these algo.