Analysis of multinomial logit model

Question

I am trying to analyse some data using a multinomial logit model, and I have a few questions regarding its interpretation.

Essentially I have data from cells from four different tissues. Each cell can belong to one of three classes.

My dataset can be reproduced as such:

library(dplyr)

grp <- structure(list(Group = c("G1", "G2", "G3", "G1", "G2", "G3", 
"G1", "G2", "G3", "G1", "G2", "G3"), Tissue = c("T1", "T1", "T1", 
"T2", "T2", "T2", "T3", "T3", "T3", "T4", "T4", "T4"), Count = c(97L, 
39L, 96L, 1829L, 378L, 881L, 47L, 55L, 14L, 74L, 78L, 40L)), row.names = c(NA, 
-12L), class = "data.frame")

grp <- grp %>% uncount(Count)

> head(grp)
    Group Tissue
1      G1     T1
1.1    G1     T1
1.2    G1     T1
1.3    G1     T1
1.4    G1     T1
1.5    G1     T1
> table(grp)
     Tissue
Group   T1   T2   T3   T4
   G1   97 1829   47   74
   G2   39  378   55   78
   G3   96  881   14   40

Now I perform a multinomial logit regression using nnet::multinom

library(nnet)
model <- multinom(Group ~ Tissue, grp)
zvalues <- summary(model)$coefficients / summary(model)$standard.errors
pvalues <- pnorm(abs(zvalues), lower.tail=FALSE)*2

This shows a significant effect of tissue type on the group

> pvalues
    (Intercept)     TissueT2     TissueT3     TissueT4
G2 1.543861e-06 7.690904e-04 0.0001000664 0.0001125417
G3 9.426030e-01 1.505263e-06 0.0003637049 0.0129607920

I could proceed and look at the pairwise differences at each level of group and tissue, but I am wondering if there is a way of "overall" comparing different tissues.

Now, if I plot estimated marginal means using

library(emmeans)

marginals <- emmeans(model, ~ Tissue + Group)
ggplot(data.frame(marginals), aes(Group, prob, group=Tissue)) + geom_line(aes(col=Tissue))

I get

Clearly, tissues T1 and T2 show similar behaviour when compared to T3 and T4, by overall belonging less to group G2

Is there a way to formally quantifying this similarity?

Answer 1

(If you’ve never seen ANOVA as a regression, pretty much nothing in this post will make sense, so we’ll have to discuss that.)

You’re basically doing ANOVA but with the response variable being a multinomial distribution instead of normal. In ANOVA, we compare a model that always predicts the overall mean (intercept only), and a model that that uses group membership as a predictor. If the latter model has a much better fit, then you conclude that the group membership affects the outcome. This is what the F-test does.

You have the same idea but with a different response.

ANOVA fits a mode by using square loss (least squares). Multinomial logistic regression uses maximum likelihood, so we compare the likelihoods of the two models: one that always predicts the overall proportions of each group (intercepts only) and one that also uses group indicator variables as predictors. If the model with group membership variables are predictors has much higher likelihood, we conclude that the group membership affects the response. This is quite analogous you the F-test.

This is called a likelihood ratio test. I know that VGAM has machinery for fitting multinomial logistic regression models and conducting the likelihood ratio test, though I am not sure about nnet.

Answer 2

The discussion with @Dave about F-tests and likelihood ratio tests seems like a distraction. Perhaps you could test the hypothesis that two pairs of tissues (T1 & T2 and T3 & T4) have the same class probabilities. But you didn't have this hypothesis before collecting the data and making the plot; instead the results suggested the pairing of T1 with T2 and T3 with T4. And you can't — convincingly — use the same data to suggest a hypothesis and also evaluate the evidence in support of that hypothesis.

Hypothesis testing in not good at "quantifying the similarity of behavior" anyway.

So how can we communicate the similarity between the tissues in terms of their probability distributions on cell groups G1, G2 & G3? We can make a plot! Specifically, a ternary plot which (I think) is effective at representing the probability simplex on three items. (In this case we have groups G1, G2, G3; and for each tissue the estimated group probabilities form a simplex as they are non-negative and sum up to 1.)

So the ternary plot highlights well the similarity of the two pairs T1 & T2 and T3 & T4. And also suggests that T3 & T4 are more "similar" than T1 & T2 are.

I would probably stop with the figure but we can also quantify these patterns in terms of the pairwise distances between the probability distributions. I decided to use Hellinger distance.

#>       T1    T2    T3
#> T2 0.124            
#> T3 0.287 0.289      
#> T4 0.209 0.235 0.086