2
$\begingroup$

Updated

I have been developing a logistic regression model based on retrospective data from a national trauma database of head injury in the UK. The key outcome is 30 day mortality (denoted as "Survive" measure). Other measures with published evidence of significant effect on outcome in previous studies include:

Yeardecimal - Date of procedure (expressed as decimal of year) = 1994-2013
Age - Age of patient = 16.0-101.5
ISS - Injury Severity Score = 16-75
Sex - Gender of patient = Male or Female
inctoCran - Time from head injury to craniotomy in minutes = 0-2880 (After 2880 minutes is defined as a separate diagnosis)
Mechanism - Mechanism of injury = Fall <2m, Fall >2m, Shooting/stabbing, RTC (Road Traffic Collision), Other
neuroFirst - Location of first admission (Neurosurgical Unit) = NSU vs. Non-NSU
rcteye - Pupil reactivity = NA / Both unreactive = O, 1 reactive = 1, both reactive = 2
rcteyeYN - dummy = 0 or 1 for presence or absence of data
GCS - Glasgow Coma Scale = 3-15
GCSYN - dummy = 0 or 1 for presence or absence of data

Dummy variables were included to enable a larger sample size where the majority of cases were excluding GCS and rcteye variables (missing not at random).

In order to test for interactions, initially I ran the following:

rcs.ASDH<-lrm(formula = Survive ~ (rcs(Age) + rcs(ISS) + 
     neuroFirst + Mechanism + rcteye + rcteyeYN + GCS + GCSYN + rcs(Yeardecimal))^2, data = ASDH_Paper1.1)

but when I did I got the following error:

singular information matrix in lrm.fit (rank= 151 ).  Offending variable(s):
GCSYN * Yeardecimal''' GCSYN * Yeardecimal' GCSYN * Yeardecimal GCS * Yeardecimal''' GCS * Yeardecimal GCS * GCSYN rcteyeYN * Yeardecimal''' rcteyeYN * Yeardecimal'' rcteyeYN * Yeardecimal rcteyeYN * GCSYN rcteye * Yeardecimal''' rcteye * Yeardecimal rcteye * rcteyeYN Mechanism=RTC * Yeardecimal''' Mechanism=Other * Yeardecimal''' Mechanism=Fall > 2m * Yeardecimal''' Mechanism=Shooting / Stabbing * Yeardecimal Mechanism=RTC * Yeardecimal Mechanism=Other * Yeardecimal Mechanism=Fall > 2m * Yeardecimal neuroFirst * Yeardecimal ISS'' * Yeardecimal''' ISS * Yeardecimal''' ISS'' * Yeardecimal'' ISS'' * Yeardecimal ISS' * Yeardecimal ISS * Yeardecimal ISS'' * GCSYN ISS'' * rcteyeYN ISS'' * Mechanism=RTC Age'' * Yeardecimal''' Age'' * Yeardecimal'' Age''' * Yeardecimal' Age''' * Yeardecimal Age'' * Yeardecimal Age' * Yeardecimal Age * Yeardecimal Age'' * GCSYN Age''' * rcteyeYN 
Error in lrm(formula = Survive ~ (rcs(Age) + rcs(ISS) + neuroFirst + Mechanism +  : 
  Unable to fit model using “lrm.fit”

The only way I could run the model is with an adjustment. Yeardecimal is excluded from any interaction as is the interaction of GCS:GCSYN and rcteye:rcteyeYN which produced the same error as written above. It made sense to exclude the interactions between a variable and its missing dummy but I am not sure what to do about Yeardecimal:

rcs.ASDH<-lrm(formula = Survive ~ (rcs(Age) + rcs(ISS) + 
     neuroFirst + Mechanism + rcteye + rcteyeYN) * (rcs(Age) + rcs(ISS) + 
     neuroFirst + Mechanism + GCS + GCSYN) + rcs(Yeardecimal), data = ASDH_Paper1.1)

From this model the following interactions were identified with an anova output:

> anova(rcs.ASDH)
                Wald Statistics          Response: Survive 

 Factor                                                Chi-Square d.f. P     
 Age  (Factor+Higher Order Factors)                    130.42      52  <.0001
  All Interactions                                      78.68      48  0.0034
  Nonlinear (Factor+Higher Order Factors)               46.53      39  0.1901
 ISS  (Factor+Higher Order Factors)                    181.65      42  <.0001
  All Interactions                                      52.43      39  0.0738
  Nonlinear (Factor+Higher Order Factors)               55.01      28  0.0017
 neuroFirst  (Factor+Higher Order Factors)              37.68      16  0.0017
  All Interactions                                      11.54      15  0.7136
 Mechanism  (Factor+Higher Order Factors)               63.72      52  0.1277
  All Interactions                                      58.35      48  0.1455
 rcteye  (Factor+Higher Order Factors)                 242.07      15  <.0001
  All Interactions                                      19.39      14  0.1507
 rcteyeYN  (Factor+Higher Order Factors)               204.58      15  <.0001
  All Interactions                                      29.88      14  0.0079
 GCS  (Factor+Higher Order Factors)                    162.81      15  <.0001
  All Interactions                                      11.62      14  0.6365
 GCSYN  (Factor+Higher Order Factors)                   94.50      15  <.0001
  All Interactions                                      41.74      14  0.0001
 Yeardecimal                                            51.96       4  <.0001
  Nonlinear                                             10.27       3  0.0164
 Age * ISS  (Factor+Higher Order Factors)               11.90      12  0.4534
  Nonlinear                                              9.40      11  0.5851
  Nonlinear Interaction : f(A,B) vs. AB                  9.40      11  0.5851
  f(A,B) vs. Af(B) + Bg(A)                               7.96       6  0.2411
  Nonlinear Interaction in Age vs. Af(B)                 8.75       9  0.4605
  Nonlinear Interaction in ISS vs. Bg(A)                 8.58       8  0.3790
 Age * neuroFirst  (Factor+Higher Order Factors)         2.66       4  0.6166
  Nonlinear                                              2.05       3  0.5624
  Nonlinear Interaction : f(A,B) vs. AB                  2.05       3  0.5624
 Age * Mechanism  (Factor+Higher Order Factors)         17.58      16  0.3493
  Nonlinear                                             13.82      12  0.3127
  Nonlinear Interaction : f(A,B) vs. AB                 13.82      12  0.3127
 Age * GCS  (Factor+Higher Order Factors)                6.24       4  0.1819
  Nonlinear                                              3.89       3  0.2741
  Nonlinear Interaction : f(A,B) vs. AB                  3.89       3  0.2741
 Age * GCSYN  (Factor+Higher Order Factors)             20.11       4  0.0005
  Nonlinear                                              8.86       3  0.0312
  Nonlinear Interaction : f(A,B) vs. AB                  8.86       3  0.0312
 ISS * neuroFirst  (Factor+Higher Order Factors)         3.23       3  0.3571
  Nonlinear                                              0.87       2  0.6480
  Nonlinear Interaction : f(A,B) vs. AB                  0.87       2  0.6480
 ISS * Mechanism  (Factor+Higher Order Factors)         23.95      12  0.0206
  Nonlinear                                             20.66       8  0.0081
  Nonlinear Interaction : f(A,B) vs. AB                 20.66       8  0.0081
 ISS * GCS  (Factor+Higher Order Factors)                0.77       3  0.8570
  Nonlinear                                              0.42       2  0.8102
  Nonlinear Interaction : f(A,B) vs. AB                  0.42       2  0.8102
 ISS * GCSYN  (Factor+Higher Order Factors)              6.53       3  0.0886
  Nonlinear                                              2.35       2  0.3085
  Nonlinear Interaction : f(A,B) vs. AB                  2.35       2  0.3085
 neuroFirst * Mechanism  (Factor+Higher Order Factors)   2.45       4  0.6533
 neuroFirst * GCS  (Factor+Higher Order Factors)         0.00       1  0.9726
 neuroFirst * GCSYN  (Factor+Higher Order Factors)       1.39       1  0.2382
 Mechanism * GCS  (Factor+Higher Order Factors)          0.10       4  0.9987
 Mechanism * GCSYN  (Factor+Higher Order Factors)        1.74       4  0.7828
 Age * rcteye  (Factor+Higher Order Factors)             8.66       4  0.0702
  Nonlinear                                              7.29       3  0.0633
  Nonlinear Interaction : f(A,B) vs. AB                  7.29       3  0.0633
 ISS * rcteye  (Factor+Higher Order Factors)             4.18       3  0.2424
  Nonlinear                                              1.49       2  0.4744
  Nonlinear Interaction : f(A,B) vs. AB                  1.49       2  0.4744
 neuroFirst * rcteye  (Factor+Higher Order Factors)      0.10       1  0.7460
 Mechanism * rcteye  (Factor+Higher Order Factors)       3.44       4  0.4867
 rcteye * GCS  (Factor+Higher Order Factors)             2.30       1  0.1297
 rcteye * GCSYN  (Factor+Higher Order Factors)           2.57       1  0.1090
 Age * rcteyeYN  (Factor+Higher Order Factors)           7.23       4  0.1242
  Nonlinear                                              7.23       3  0.0649
  Nonlinear Interaction : f(A,B) vs. AB                  7.23       3  0.0649
 ISS * rcteyeYN  (Factor+Higher Order Factors)           2.47       3  0.4814
  Nonlinear                                              0.11       2  0.9462
  Nonlinear Interaction : f(A,B) vs. AB                  0.11       2  0.9462
 neuroFirst * rcteyeYN  (Factor+Higher Order Factors)    0.12       1  0.7280
 Mechanism * rcteyeYN  (Factor+Higher Order Factors)     1.81       4  0.7701
 rcteyeYN * GCS  (Factor+Higher Order Factors)           3.70       1  0.0543
 rcteyeYN * GCSYN  (Factor+Higher Order Factors)         8.74       1  0.0031
 TOTAL NONLINEAR                                       102.74      64  0.0015
 TOTAL INTERACTION                                     178.52     103  <.0001
 TOTAL NONLINEAR + INTERACTION                         241.87     111  <.0001
 TOTAL                                                 889.91     123  <.0001

The summary function revealed the following results:

             Effects              Response : Survive 

 Factor                                    Low    High   Diff. Effect       S.E.   Lower 0.95 Upper 0.95    
 Age                                         37.6   72.0 34.40         0.15   0.38   -0.58      8.900000e-01
  Odds Ratio                                 37.6   72.0 34.40         1.16     NA    0.56      2.430000e+00
 ISS                                         20.0   26.0  6.00        -1.34   0.31   -1.95     -7.400000e-01
  Odds Ratio                                 20.0   26.0  6.00         0.26     NA    0.14      4.800000e-01
 neuroFirst                                   0.0    1.0  1.00        -0.23   0.37   -0.95      5.000000e-01
  Odds Ratio                                  0.0    1.0  1.00         0.80     NA    0.39      1.650000e+00
 rcteye                                       0.0    2.0  2.00         3.20   0.50    2.22      4.170000e+00
  Odds Ratio                                  0.0    2.0  2.00        24.41     NA    9.24      6.452000e+01
 rcteyeYN                                     0.0    1.0  1.00        -3.34   0.44   -4.21     -2.480000e+00
  Odds Ratio                                  0.0    1.0  1.00         0.04     NA    0.01      8.000000e-02
 GCS                                          0.0   12.0 12.00         1.94   0.49    0.98      2.890000e+00
  Odds Ratio                                  0.0   12.0 12.00         6.94     NA    2.67      1.799000e+01
 GCSYN                                        0.0    1.0  1.00        -1.32   0.45   -2.20     -4.400000e-01
  Odds Ratio                                  0.0    1.0  1.00         0.27     NA    0.11      6.400000e-01
 Yeardecimal                               2005.5 2012.4  6.85         0.20   0.12   -0.03      4.400000e-01
  Odds Ratio                               2005.5 2012.4  6.85         1.22     NA    0.97      1.550000e+00
 Mechanism - Fall > 2m:Fall < 2m              1.0    2.0    NA        -0.89   0.35   -1.58     -2.000000e-01
  Odds Ratio                                  1.0    2.0    NA         0.41     NA    0.21      8.200000e-01
 Mechanism - Other:Fall < 2m                  1.0    3.0    NA         0.25   0.42   -0.58      1.080000e+00
  Odds Ratio                                  1.0    3.0    NA         1.28     NA    0.56      2.930000e+00
 Mechanism - RTC:Fall < 2m                    1.0    4.0    NA        -0.68   0.43   -1.52      1.700000e-01
  Odds Ratio                                  1.0    4.0    NA         0.51     NA    0.22      1.190000e+00
 Mechanism - Shooting / Stabbing:Fall < 2m    1.0    5.0    NA        18.97 116.63 -209.63      2.475600e+02
  Odds Ratio                                  1.0    5.0    NA 172906690.96     NA    0.00     3.272814e+107

Adjusted to: Age=54.2 ISS=25 neuroFirst=0 Mechanism=Fall < 2m rcteye=1 rcteyeYN=0 GCS=3 GCSYN=0

Remaining questions are:

1 - Is my dummy variable treatment for variables missing not at random appropriate, including the exclusion of interactions with the main term?

2 - Can I resolve the issues with assessing interaction of the Yeardecimal term?

3 - Should I exclude non-significant interaction terms? I read that exclusion only of a "chunk" is advised - Including the interaction but not the main effects in a model

4 - Is the odds ratio for each variable the "Effect" column? If so, is this the OR between the lowest and highest value of each variable?

Improve this question
$\endgroup$
2
  • $\begingroup$ Question you should be asking is how to present the spline fit to allow someone to understand & use your model. rcspline.restate can be useful in this respect. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Dec 8, 2014 at 12:44
  • $\begingroup$ Consider adding to the question or asking a new one rather than editing so drastically that nothing remains of the original. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Dec 9, 2014 at 10:23

1 Answer 1

Reset to default
3
$\begingroup$

You are assuming linearity for time and age. This is unlikely to be correct. You have not included a possible nonlinear interaction between age and sex. You don't need x=TRUE, y=TRUE if you are not bootstrapping or using robust covariance matrix estimation.

To answer your questions, a graph is the best summary of a partial effect, and you can include a small graph in an abstract, which also makes the abstract more likely to be read.

The other approach is to estimate odds ratios and their confidence limits. The summary function (full name: summary.rms) makes this easy. By default it gives you the inter-quartile-range odds ratios. You can easily override the covariate settings to ask for odds ratios for any two levels of the covariate you choose. The help file for summary.rms has lots of examples.

Your limiting sample size is 629, so you can reliably fit roughly $\frac{629}{15}$ parameters or 42. So you are in pretty good shape in terms of avoiding overfitting, otherwise you might need to greatly emphasize rigorous internal validation using the bootstrap. It is still worth doing this to obtain a smooth overfitting-corrected calibration curve using the calibrate function.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Post has been updated to reflect your feedback with further questions. Many thanks for your help so far. $\endgroup$
    – Dan Fountain
    Dec 8, 2014 at 21:07
  • $\begingroup$ It is not correct to use indicator variables for missing values. Use multiple imputation instead. I wouldn't include all two-way interactions but would talk to subject matter experts about what interactions would make sense. $\endgroup$
    – Frank Harrell
    Dec 9, 2014 at 21:03
  • $\begingroup$ Clear on interactions. My concern is that for these missing values not missing at random, there is very little data for the sample. For instance, pupil reactivity was only recorded partially from 2007 in the database, resulting in only 44% of cases with values. Similarly, only 63% of cases have a recorded GCS value. If multiple imputation is possible, are standard approaches such as in your textbook feasible for application here? $\endgroup$
    – Dan Fountain
    Dec 10, 2014 at 10:43
  • 1
    $\begingroup$ Yes. I recommend predictive mean matching. You can look at the R Hmisc package aregImpute function. There is no amount of missing data that would make it better to drop variables or observations entirely. The indicator variable method fails even with a small fraction of missings. Of course your model will be limited when missingness is high, e.g., high standard errors and more dependence on assumptions. $\endgroup$
    – Frank Harrell
    Dec 10, 2014 at 11:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.