Categorical response variable prediction

Question

I have the following kind of data (coded in R):

v.a = c('cat', 'dog', 'dog', 'goat', 'cat', 'goat', 'dog', 'dog')
v.b = c(1, 2, 1, 2, 1, 2, 1, 2)
v.c = c('blue', 'red', 'blue', 'red', 'red', 'blue', 'yellow', 'yellow')
set.seed(12)
v.d = rnorm(8)
aov(v.a ~ v.b + v.c + v.d) # Error

I would like to know if the value of v.b or the value of v.c has any ability to predict the value of v.a. I would run an ANOVA (as shown above) but I think it does not make any sense since my response variable is not ordinal (it is categorical). What should I do?

Answer 1

You could use ANY classifier. Including Linear Discriminants, multinomial logit as Bill pointed out, Support Vector Machines, Neural Nets, CART, random forest, C5 trees, there are a world of different models that can help you predict $v.a$ using $v.b$ and $v.c$. Here is an example using the R implementation of random forest:

# packages
library(randomForest)

#variables
v.a= c('cat','dog','dog','goat','cat','goat','dog','dog')
v.b= c(1,2,1,2,1,2,1,2)
v.c= c('blue', 'red', 'blue', 'red', 'red', 'blue', 'yellow', 'yellow')

# model fit
# note that you must turn the ordinal variables into factor or R wont use
# them properly
model <- randomForest(y=as.factor(v.a),x=cbind(v.b,as.factor(v.c)),ntree=10)

#plot of model accuracy by class
plot(model)

# model confusion matrix
model$confusion

Clearly these variables don't show a strong relation.

Answer 2

_{This is a more a partial practical answer, but it works for me to do some exercises before getting deeply into theory}.

This ats.ucla.edu link is a reference that might help beggining to understand about multinomial logistic regression (as pointed out by Bill), in a more practical way.
It presents reproducible code to understand function multinom from nmet package in R and also gives a briefing about outputs interpretation.

Consider this code:

va = c('cat','dog','dog','goat','cat','goat','dog','dog') 
     # cat will be the outcome baseline
vb = c(1,2,1,2,1,2,1,2)
vc = c('blue','red','blue','red','red','blue','yellow','yellow') 
     # blue will be the vc predictor baseline
set.seed(12)
vd = round(rnorm(8),2)

data = data.frame(cbind(va,vb,vc,vd))

library(nnet)
fit <- multinom(va ~ as.numeric(vb) + vc + as.numeric(vd), data=data)

# weights:  18 (10 variable)
initial  value 8.788898 
iter  10 value 0.213098
iter  20 value 0.000278
final  value 0.000070 
converged

fit

Call:
multinom(formula = va ~ as.numeric(vb) + vc + as.numeric(vd), 
    data = data)

Coefficients:
     (Intercept) as.numeric(vb)     vcred  vcyellow as.numeric(vd)
dog    -1.044866       120.3495 -6.705314  77.41661      -21.97069
goat   47.493155       126.4840 49.856414 -41.46955      -47.72585

Residual Deviance: 0.0001656705 
AIC: 20.00017 

This is how you can interpret the log-linear fitted multinomial logistic model:

\begin{align} \ln\left(\frac{P(va={\rm cat})}{P(va={\rm dog})}\right) &= b_{10} + b_{11}vb + b_{12}(vc={\rm red}) + b_{13}(vc={\rm yellow}) + b_{14}vd \\ &\ \\ \ln\left(\frac{P(va={\rm cat})}{P(va={\rm goat})}\right) &= b_{20} + b_{21}vb + b_{22}(vc={\rm red}) + b_{23}(vc={\rm yellow}) + b_{24}vd \end{align}

Here is an excerpt about how the model parameters can be interpreted:

• A one-unit increase in the variable vd is associated with the decrease in the log odds of being "dog" vs. "cat" in the amount of 21.97069 ($b_{14}$).

the same logic for the second line but, considering "goat" vs. "cat" with ($b_{24}$=-47.72585).

• The log odds of being "dog" vs. "cat" will increase by 6.705314 if moving from vc="blue" to vc="red"($b_{12}$).

.....

There is much more in the article, but I thought this part to be the core.

---

Reference:

R Data Analysis Examples: Multinomial Logistic Regression. UCLA: Statistical Consulting Group.
from http://www.ats.ucla.edu/stat/r/dae/mlogit.htm (accessed November 05, 2013).