I have the following kind of data (coded in R):

v.a = c('cat', 'dog', 'dog', 'goat', 'cat', 'goat', 'dog', 'dog')
v.b = c(1, 2, 1, 2, 1, 2, 1, 2)
v.c = c('blue', 'red', 'blue', 'red', 'red', 'blue', 'yellow', 'yellow')
v.d = rnorm(8)
aov(v.a ~ v.b + v.c + v.d) # Error

I would like to know if the value of v.b or the value of v.c has any ability to predict the value of v.a. I would run an ANOVA (as shown above) but I think it does not make any sense since my response variable is not ordinal (it is categorical). What should I do?

  • 5
    $\begingroup$ Learn about multinomial logit. Here are two, free online books by Kenneth Train (elsa.berkeley.edu/books/choice.html and elsa.berkeley.edu/books/choice2.html). I think those are graduate level books. Or just google around for "multinomial logit." $\endgroup$
    – Bill
    Nov 5 '13 at 13:59
  • $\begingroup$ @Bill this looks like a good start on an answer. Please consider expanding a little bit on what multinomial logit models are, and posting it as an answer. $\endgroup$
    – Glen_b
    Nov 5 '13 at 15:03

You could use ANY classifier. Including Linear Discriminants, multinomial logit as Bill pointed out, Support Vector Machines, Neural Nets, CART, random forest, C5 trees, there are a world of different models that can help you predict $v.a$ using $v.b$ and $v.c$. Here is an example using the R implementation of random forest:

# packages

v.a= c('cat','dog','dog','goat','cat','goat','dog','dog')
v.b= c(1,2,1,2,1,2,1,2)
v.c= c('blue', 'red', 'blue', 'red', 'red', 'blue', 'yellow', 'yellow')

# model fit
# note that you must turn the ordinal variables into factor or R wont use
# them properly
model <- randomForest(y=as.factor(v.a),x=cbind(v.b,as.factor(v.c)),ntree=10)

#plot of model accuracy by class

enter image description here

# model confusion matrix

Clearly these variables don't show a strong relation.

  • $\begingroup$ @JEquihua Could you please tell me a bit more about what is a "tree" and what is the meaning of the output(confusion matrix and the plot). Thanks a lot! $\endgroup$
    – Remi.b
    Nov 8 '13 at 8:58
  • $\begingroup$ I will. I'm very busy, give me a little time. @Remi.b $\endgroup$
    – JEquihua
    Nov 19 '13 at 15:59

This is a more a partial practical answer, but it works for me to do some exercises before getting deeply into theory.

This ats.ucla.edu link is a reference that might help beggining to understand about multinomial logistic regression (as pointed out by Bill), in a more practical way.
It presents reproducible code to understand function multinom from nmet package in R and also gives a briefing about outputs interpretation.

Consider this code:

va = c('cat','dog','dog','goat','cat','goat','dog','dog') 
     # cat will be the outcome baseline
vb = c(1,2,1,2,1,2,1,2)
vc = c('blue','red','blue','red','red','blue','yellow','yellow') 
     # blue will be the vc predictor baseline
vd = round(rnorm(8),2)

data = data.frame(cbind(va,vb,vc,vd))

fit <- multinom(va ~ as.numeric(vb) + vc + as.numeric(vd), data=data)

# weights:  18 (10 variable)
initial  value 8.788898 
iter  10 value 0.213098
iter  20 value 0.000278
final  value 0.000070 


multinom(formula = va ~ as.numeric(vb) + vc + as.numeric(vd), 
    data = data)

     (Intercept) as.numeric(vb)     vcred  vcyellow as.numeric(vd)
dog    -1.044866       120.3495 -6.705314  77.41661      -21.97069
goat   47.493155       126.4840 49.856414 -41.46955      -47.72585

Residual Deviance: 0.0001656705 
AIC: 20.00017 

This is how you can interpret the log-linear fitted multinomial logistic model:

\begin{align} \ln\left(\frac{P(va={\rm cat})}{P(va={\rm dog})}\right) &= b_{10} + b_{11}vb + b_{12}(vc={\rm red}) + b_{13}(vc={\rm yellow}) + b_{14}vd \\ &\ \\ \ln\left(\frac{P(va={\rm cat})}{P(va={\rm goat})}\right) &= b_{20} + b_{21}vb + b_{22}(vc={\rm red}) + b_{23}(vc={\rm yellow}) + b_{24}vd \end{align}

Here is an excerpt about how the model parameters can be interpreted:

  • A one-unit increase in the variable vd is associated with the decrease in the log odds of being "dog" vs. "cat" in the amount of 21.97069 ($b_{14}$).

the same logic for the second line but, considering "goat" vs. "cat" with ($b_{24}$=-47.72585).

  • The log odds of being "dog" vs. "cat" will increase by 6.705314 if moving from vc="blue" to vc="red"($b_{12}$).


There is much more in the article, but I thought this part to be the core.


R Data Analysis Examples: Multinomial Logistic Regression. UCLA: Statistical Consulting Group.
from http://www.ats.ucla.edu/stat/r/dae/mlogit.htm (accessed November 05, 2013).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.