2
$\begingroup$

I was wondering what are good ways to visualize the errors made by a multinomial logistic regression model?

I can compute the class probabilities and I have the a test set with the real classes and a vector of predicted classes. What are the most useful ways to visualize the performance of my model?

In addition, I was wondering whether you can calculate some kind of residual out of it. If so, should I then use the class probabilities to calculate the residual or the predictions (as in just picking the class with the highest probability). In order to clarify my statement consider the fact that I have 3 classes.

If I use class probability to construct some kind of residual, for the first observation I would predict the following class probabilities. Given that this observation is of class A, we then get a residual of:

(0.25-1)^2 + (0.33-0)^2 + (0.42-0)^2

A: 0.25
B: 0.33
C: 0.42

However if I use the predictions I will get:

(0-1)^2 + (0-0)^2 + (1-0)^2

What would be a better option and what are good informative graphs for multinomial models?

I have already created a confusion matrix with a heatmap.

Update:

Code for rolando2. This is what I meant with a confusion matrix with heatmap.

library(ggplot2)
library(data.table)

# Generating fake confusion matrix
dt <- data.table(Prediction = c("A", "A", "B", "B"), 
             Reference = c("A", "B", "A", "B"),
             Value = c(10, 11, 15, 20))

ggplot(dt, aes(x = Prediction, y = Reference)) +  
geom_tile(aes(fill = Value)) +
theme_bw() +
geom_text(aes(x = Prediction, y = Reference, label = Value), colour = "white")

Confusion Matrix with a heatmap

$\endgroup$
  • $\begingroup$ "Confusion matrix with heatmap" sounds promising; care to show that? Hard to see the relevance of residuals when you have a dependent variable with 3 or more nominal categories. $\endgroup$ – rolando2 Dec 20 '15 at 20:06
  • $\begingroup$ I have included the "confusion matrix with heatmap". $\endgroup$ – Snowflake Dec 20 '15 at 20:50
  • $\begingroup$ Thanks for the code but I was actually referring to your display itself :-) Might help people give you better comments about your analysis. $\endgroup$ – rolando2 Dec 20 '15 at 22:06
  • $\begingroup$ Included the image ;). $\endgroup$ – Snowflake Dec 21 '15 at 5:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.