7
$\begingroup$

I am running an ordinal logistic regression in R and running into trouble when I include dummy variables. My model works great with my first set of predictors. Next I want to add dummy variables for each of the years represented in my dataset.

I created the dummy variables with car:recode in this manner (one statement like this for each of the 11 years)

fsd$admityear2000 <- recode(fsd$ApplicationYear ,"2000=1;else=0")

The lrm model is specified as follows

library(Design)
ddist<- datadist(fsd)
options(datadist='ddist')
m4 <- lrm(Outcome ~ relGPA + mcAvgGPA + Interview_Z + WorkHistory_years + GMAT + UGI_Gourman + admityear1999 + admityear2000 + admityear2001 + admityear2002 + admityear2003 + admityear2004 + admityear2005 + admityear2006 + admityear2007 + admityear2008 + admityear2009, data=fsd)

(sorry for all of the other random variables, but I don't want to introduce confusion by changing my code)

I get the error

singular information matrix in lrm.fit (rank= 22 ).  Offending variable(s):
admityear2009 admityear2000 admityear1999 
Error in lrm(Outcome ~ relGPA + mcAvgGPA + Interview_Z + WorkHistory_years +  : 
  Unable to fit model using “lrm.fit”

I understand that including all options of a dummy variable over-defines the model, but I get the error whether I include all 11 years or just 10.

I found a suggestion here to set the penalty parameter of lrm to a small positive value. Setting it to 1 or 5 changes the error such that it only names one of the variables as offending. The error doesn't go away even with penalty=100.

I'm pretty new to R, but loving the freedom so far. Thanks for any help!

Responses and Lessons

  • Factors are awesome and I can't believe I didn't notice them earlier. Man that cleans up my code a lot. Thanks!
  • My DV, 'Outcome' is indeed ordinal and after making it a factor(), I also made it ordered().
  • The str() command is also awesome and this is what my data now looks like (with some of the non-relevant variables omitted)

output:

str(fsd)     
Outcome      : Ord.factor w/ 3 levels "0"<"1"<"2"
relGPA       : num  
mcAvgGPA     : num 
admitschool  : Factor w/ 4 levels "1","2","3","4"
appyear      : Factor w/ 11 levels "1999","2000",..
  • both lrm() and polr() now run successfully, and they both deal with appyear by dropping some values of the factor. lrm() drops 1999, 2000, and 2001 while polr() just drops 1999 and 2000. lrm() gives no warnings while polr() says "design appears to be rank-deficient, so dropping some coefs." This is an improvement, but I still don't understand why more than one value needs to be dropped. xtabs shows that there isn't full seperation right?

output:

xtabs(~fsd$appyear + fsd$Outcome)
           fsd$Outcome
fsd$appyear    0    1    2
       1999 1207  123  418
       2000 1833  246  510
       2001 1805  294  553
       2002 1167  177  598
       2003 4070  158 1076
       2004 2803  106 1138
       2005 3749  513 2141
       2006 4429  519 2028
       2007 6134  670 1947
       2008 7446  662 1994
       2009 4411   86 1118
$\endgroup$
  • $\begingroup$ Using the Hmisc:redun() function I can confirm that the variables are not redundant so long as I only include 10 of the 11 $\endgroup$ – Sam Swift Feb 23 '11 at 22:27
  • $\begingroup$ I found another suggestion to change 'tol' to a value smaller than its default 1e-7. The results are quite similar to changing 'penalty' above. Changing it to smaller values reduces the 'Offending variables' list to only 'admityear2000' $\endgroup$ – Sam Swift Feb 23 '11 at 22:27
  • 1
    $\begingroup$ a week later, I have figured out more precisely what the problem was. I was using two factors, admitting school and admit-year. For some years we only had data from a single admitting school, so the levels for that year and that school were redundant. Understanding that makes me comfortable having them drop out of the model. $\endgroup$ – Sam Swift Mar 1 '11 at 23:06
7
$\begingroup$

Creating dummy variables should not be necessary. You should just use factors when modeling in R.

admityear <- factor(admityear) 
m4 <- lrm(Outcome ~ relGPA + mcAvgGPA + Interview_Z + WorkHistory_years + 
                       GMAT + UGI_Gourman + admityear, data=fsd)

If the singular condition still persists, then you have multicollinearity and need to try dropping other variables. (I would be suspicious of WorkHistory_years.) I also don't see anything ordinal about that model. Ordinal logistic regression in the rms package (or the no longer actively supported Design package) is done with polr(). And it would be really helpful to see the results from str(fasd).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Assuming Outcome is an ordinal factor, I believe lmr() will treat it as such. At least the documentation for lmr() and lmr.fit() seem to indicate that it will be. <br />Assuming your categorical variable is set up as a factor, as suggested by DWin, and your response variable is set up as an ordered factor, the inability to estimate the model may be occurring because of separation. Basically, some categories of admityear may perfectly `explain' a particular outcome. $\endgroup$ – Jason Morgan Feb 24 '11 at 4:35
  • $\begingroup$ Thanks so much to both of you. I learned about 5 new things already. I'll add responses and lessons to my question $\endgroup$ – Sam Swift Feb 24 '11 at 14:46
  • $\begingroup$ There may be separation when cross-classifying by two or more factors that doesn't show up on a simple predictor by Outcome tabulation. You would need to look at multi-way tables to find it. $\endgroup$ – DWin Feb 25 '11 at 4:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.