should I use n-1 dummies variables or all variables for a multinomial logistic regression?

Question

Recently I have been working with gut microbiome data, like abundance and its metabolic content (but for purposes of the question this may be indifferent). I'm inexpert in the field of multinomial logistic regression and also in how to report it in a scientific paper.

I'm working in R (the one_hot() to convert my categorical independent variables to one hot and multinom() for generate my model), with a sample of 402 individuals (213 for healthy or control or reference group, 56 group 1, 50 group unhealthy 2, 44 group unhealthy 3, 39 group unhealthy 4), the scope of my analysis is to identify those interactions of bacteria (3 types of interaction: cooperative, competitive, no interaction) that are related to each group and how much does each bacterial interaction contribute to describing each group. I used the 5 groups of individuals as response or dependent variable, and the bacterial interactions as predictors or independent variables. And for my final model I just keep those independent variables that has a p<0.05

My issue: In the beginning, I set it as an independent categorical variable without converting it to dummies in the "multinom()" function. For that model, I saw that "multinom(status~. , data = df_categorical) " generates 2 dummies variables (n-1) for my categorical variable with 3 categories. I got a 73% of accuracy for that model. Then I select a different set of 2 dummies and I got 74% of accuracy with "multinom(status~. , data = df_onehot_coperative_and_competitive)". Then I tried using the 3 dummies in my model and a got 75% with "multinom(status~. , data = df_onehot_cop_com_non)". The way I see the accuracy is by dividing the number of right classified individuals by my model by the total number of individuals. I don't use all my data to training the model and all the data to generate the predictions because I just want to describe my sample data by the independent variables.

I found on the web that the right thing is to use n-1 dummies to generate a model, but I was expecting to get the same accuracy in every selection of dummies.

My questions:

1.- am I getting the wrong result? or why do I get a different accuracy?

2.- should I keep the 3 dummies for the best accuracy?

3.- If I use just 2 of my 3 dummies variables, how can I interpret the beta coefficients respecting the missing dummy? (because I want to interpret the model based on its beta)

4.- if I got some not statistical significant dummies (2 of the 3) should I remove them? should I generate a new model with just the significant dummies?

If you could attach your source I would be very grateful, but it is not necessary

Answer 1

Before you worry about the one-hot coding, you should evaluate a few other issues with your model.

213 for healthy or control or reference group, 56 group 1, 50 group unhealthy 2, 44 group unhealthy 3, 39 group unhealthy 4

First, a multinomial model is probably not what you want. That's appropriate for an unordered multi-level categorical outcome. You have a clear ordering of your 5 outcome groups. That's best examined with an ordinal-outcome model, for example a proportional-odds logistic model.

I don't use all my data to training the model and all the data to generate the predictions because I just want to describe my sample data by the independent variables.

Second, with only 400 cases you should use all your data for training and then validate your modeling process by bootstrapping. See this post, for example.

the accuracy is by dividing the number of right classified individuals by my model by the total number of individuals

Third, you probably shouldn't be using accuracy as your measure of model quality. It would seem much more important to distinguish normal cases from disease cases rather then to distinguish a group 3 from a group 4 case. If you use accuracy you assume that all misclassifications have the same cost.

And for my final model I just keep those independent variables that has a p<0.05.

Fourth, removing "insignificant" individual predictors is a bad idea. See this page and many others on this site about the dangers of using such an automated process for defining your model. I suspect that has something to do with your getting different "accuracy" results from the different models. Those differences are very small in any event, representing differences of only a handful of cases among your models.

If I use just 2 of my 3 dummies variables, how can I interpret the beta coefficients respecting the missing dummy?

Fifth, the standard R coding is the safest way to go. The intercept of your model then represents the situation for your categorical predictor at its reference value. The 2 beta coefficients are the differences of the other 2 levels from the reference level. There is no beta coefficient for the reference level. Trying to code with 3 dummy variables will tend to get you into trouble. Depending on the rest of your model, you might be able to use 3 dummies if you omit the intercept but I wouldn't recommend trying that.

if I got some not statistical significant dummies (2 of the 3) should I remove them? should I generate a new model with just the significant dummies?

Sixth, absolutely not, and for reasons that go beyond the general cautions above about "insignificant" predictors. The displayed beta coefficients for non-reference levels are for their differences from the reference level. So the apparent "significance" of one beta coefficient of a multi-category predictor will depend on your choice of reference level.

You need to keep all levels of your multi-category predictor and evaluate the "significance" of all levels together. You can do that with a "chunk" (Wald) test on all levels of the predictor, or with a likelihood-ratio test between a model that includes all the levels and a model that includes none of them.

Most of these, and many other issues in regression modeling, are covered by Frank Harrell's course notes and book. His rms package provides the tools you need for ordinal logistic regression, combined anova() tests on all levels of a predictor, and for bootstrap validation and calibration of your modeling process.