0
$\begingroup$

I'm trying to pick a classification model for my dataset that looks like this:

ID_Variable | V1 - V2 - ... - V6 | Current_State | Next_State

V1 to V6 are continuous variables with information like sales, growth rate and distribution etc. Current_State is a categorical variable with 10 levels (Nascent, Peaked, Prime etc.) Next_State is same as Current_State - factor with 10 levels

A sample row(s) look like this:

ID Variable           V1        V2    V3    V4   V5   V6  Current_State  Next_State
Product_1_Phase_1    115.45    0.10  0.05  1    0.5   4        '7'         '8'
Product_2_Phase_3    20167.98 -0.32 -0.33  3.66 0.25  6        '5'        '8'

I want to predict the Next_State of each ID_Variable by training a classification model.

Since the number of data points that I have is really less ~250 Rows, based on my research on the topic, I need to look into High Bias - Low Variance models.

I tried Naive Bayes classification in R with an accuracy of 41%. Also, I get good Specificity for each class, Sensitivity is all over the place.

The output of confusion matrix :

|          | Sensitivity| Specificity| Pos Pred Value| Neg Pred Value| Precision|    Recall|        F1| Prevalence| Detection Rate| Detection Prevalence| Balanced Accuracy|
|:---------|-----------:|-----------:|--------------:|--------------:|---------:|---------:|---------:|----------:|--------------:|--------------------:|-----------------:|
|Class: 1  |   0.3333333|   0.9827586|      0.5000000|      0.9661017| 0.5000000| 0.3333333| 0.4000000|  0.0491803|      0.0163934|            0.0327869|         0.6580460|
|Class: 2  |   1.0000000|   0.9649123|      0.6666667|      1.0000000| 0.6666667| 1.0000000| 0.8000000|  0.0655738|      0.0655738|            0.0983607|         0.9824561|
|Class: 3  |   0.4285714|   0.9629630|      0.6000000|      0.9285714| 0.6000000| 0.4285714| 0.5000000|  0.1147541|      0.0491803|            0.0819672|         0.6957672|
|Class: 4  |   0.6250000|   0.9245283|      0.5555556|      0.9423077| 0.5555556| 0.6250000| 0.5882353|  0.1311475|      0.0819672|            0.1475410|         0.7747642|
|Class: 5  |   0.0000000|   0.9824561|      0.0000000|      0.9333333| 0.0000000| 0.0000000|       NaN|  0.0655738|      0.0000000|            0.0163934|         0.4912281|
|Class: 6  |   0.0000000|   0.9818182|      0.0000000|      0.9000000| 0.0000000| 0.0000000|       NaN|  0.0983607|      0.0000000|            0.0163934|         0.4909091|
|Class: 7  |   0.5714286|   0.8888889|      0.4000000|      0.9411765| 0.4000000| 0.5714286| 0.4705882|  0.1147541|      0.0655738|            0.1639344|         0.7301587|
|Class: 8  |   0.6250000|   0.7547170|      0.2777778|      0.9302326| 0.2777778| 0.6250000| 0.3846154|  0.1311475|      0.0819672|            0.2950820|         0.6898585|
|Class: 9  |   0.3333333|   0.8846154|      0.3333333|      0.8846154| 0.3333333| 0.3333333| 0.3333333|  0.1475410|      0.0491803|            0.1475410|         0.6089744|
|Class: 10 |   0.0000000|   1.0000000|            NaN|      0.9180328|        NA| 0.0000000|        NA|  0.0819672|      0.0000000|            0.0000000|         0.5000000|

Can anyone please guide in this matter. Any help is much appreciated.

$\endgroup$
  • 1
    $\begingroup$ Can you add some more detail about your problem? A few example training instances, for example, and/or a more in depth look at your results with the Naïve Bayes Classifier. $\endgroup$ – user77876 Sep 13 '17 at 11:08
  • 1
    $\begingroup$ I did add a few more details to clarify the problem. Thanks! $\endgroup$ – Harsh Gupta Sep 13 '17 at 12:48
1
$\begingroup$

It is of course always a possibility that your classes are unseparable based on your data and it might be possible that they are separable but only with much more data.

In the meantime, there are still things you can try. Naive Bayes makes strong assumptions about your variables and is incapable of detecting if some variables are more useful for prediction than others. For example, current state could be a much better indicator of next state than your other variables, but naive bayes would not detect or use that.

If you tried a simple decision tree, it would tell you if some class is much more useful than others. With ten classes and few records, you probably need to tune the pruning severity to see where it gets you the best results.

Also, decide your performance metric upfront for principled reasons and don't look at 10 different ones and go shopping for the one that supports your narrative best, that's p-hacking. With ten classes, if you don't have misclassification cost imbalance, just take accuracy.

If you need to look at precision and recall and all classes have the same importance on the class level, not record level, take macro averaged F-measure for multiple classes. That means that less prevalent classes are as important as more prevalent classes, making records in less prevalent classes more important. If you have specific weighing for the classes in mind, you can also make it a weighed macro averaged F-measure.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.