1
$\begingroup$

I was wondering if someone with experience running multinomial logistic regression could look at my data file and results, and explain why the results turned out the way it did.

The background: I've had 29 respondents complete 18 tasks where they had to choose one of 5 options given a set of 7 attributes of varying levels. My goal is to determine how their answers are impacted by the 7 attributes, and predict their probability of selecting a category given a combination of attributes.

Here is the raw data file: https://docs.google.com/spreadsheets/d/14JoKN_ZM_RG6eMzJ6jK19OgTyXxib4N4GaIjCwMSNPk/edit?usp=sharing

To explain the data file: ID and sys_RespNum are both respondent ID variables. "Version" references which version of the questionnaire they were shown, and can be ignored". "Task" indicates which one of the 18 tasks the case is about. DV is my dependent variable - 5 categories that the respondent selected for each task. Att1 to Att7 are the 7 attributes that the respondent were shown, each of varying levels:

  • Att1 has 5 levels
  • Att2 and Att3 each have 2 levels
  • Att4 has 6 levels
  • Att5 and Att6 each have 2 levels
  • Att7 has 3 levels.

The remaining variables repeat the attribute variables, but this time in binomial format. So each level of each attribute (save for the first level, removed to avoid linear dependency) is a binomial variable, in which 1 is "not present" and 2 is "present".

The distribution of the dependent variable is as follows:

  • Category 1: selected 36% of the time
  • Category 2: selected 39% of the time
  • Category 3: selected 7% of the time
  • Category 4: selected 6% of the time
  • Category 5: selected 12% of the time.

I ran a multinomial logistic regression using the nnet package in R with the following code:

multinom(DV~ATT1_2+ATT1_3+ATT1_4+ATT1_5+ATT2_2+ATT3_2+ATT4_2+ATT4_3+ATT4_4+ATT4_5+ATT4_6+ATT5_2+ATT6_2+ATT7_2+ATT7_3, mydata)

Here is the output:

    (Intercept)   ATT1.2      ATT1.3      ATT1.4      ATT1.5      ATT2.2      ATT3.2      ATT4.2      ATT4.3      ATT4.4      ATT4.5      ATT4.6      ATT5.2      ATT6.2      ATT7.2      ATT7.3                                                                    
2   9.094252    -1.6129788  -2.213684   -2.547264   -2.590407   -1.0456736  0.03926262  0.1889604   0.2776733   0.3368661   0.2842599   0.5835186   0.3125102   -0.4085997  0.4196491   0.7406358
3   2.739365    -1.2033615  -2.279925   -2.203506   -2.852359   -0.8794329  0.32243784  0.5449504   0.2952917   1.0482031   1.385338    1.9938475   0.4604817   -1.5523677  0.5580864   0.8205683
4   15.733501   -2.4776377  -2.904956   -2.358851   -4.592486   0.292845    -0.01494513 0.1362193   -1.0740628  -0.6020315  -0.9300861  -1.611219   0.3579449   -0.6106962  0.5534503   0.7170613
5   10.767058   -0.7320382  -1.127888   -1.162103   -1.365407   0.112293    -0.29885381 -0.2111886  -1.231965   -1.7296445  -0.3197262  -3.0657761  -0.3043559  0.2824785   0.3371815   0.4749809

I then calculate the probability of a category being selected using the logit rule. The problem is that for almost all types of attributes I select as independent variables, category 4 is given a probability of 99%. This is despite the fact that it was the least often selected category in respondents' answers.

Can anyone look at what I've done and tell me where I went wrong?

EDIT: The problem was that the values I was using to code the predictors were different than what they were in the data the model was based on. The binomial variables in the data were 1 or 2, the values I was using for the model were 0 or 1. I am not a smart man.

$\endgroup$
  • $\begingroup$ My concern is , why are you using a single model? From what I see, there are 18 different tasks, hence you should have 18 different models. Or am I missing something here? $\endgroup$ – show_stopper Oct 27 '14 at 23:38
  • $\begingroup$ Each task is a separate measurement for each respondent using an experimental design - so the attributes are shown in such a way that they are orthogonal and balanced across respondents. The objective is to determine what characteristics of attributes lead to selecting one category over the others, hence why the aim was to have one model. $\endgroup$ – Phil Oct 27 '14 at 23:49
  • $\begingroup$ Re-calculate the predicted probabilities paying careful attention to how you've coded the predictors. The results should match predict(mymodel, type="prob"). If they don't, & you can't see why, edit the question to show exactly what you're doing. $\endgroup$ – Scortchi Oct 28 '14 at 12:30
  • $\begingroup$ Thanks Scortchi. I just figured out what the error was, and I can officially say that I'm not a smart man. $\endgroup$ – Phil Oct 28 '14 at 16:01
  • $\begingroup$ @Phil: That's what I thought it might be - the intercepts are all huge on the logit scale. (Long experience of making similar mistakes, not smartness.) $\endgroup$ – Scortchi Oct 28 '14 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.