1
$\begingroup$

How can I determine whether one coding of a linear predictor leads to a better fit of the corresponding regression model than the other? In the following example, the restricted cubic spline coding of albumin leads to a higher chi-square value of the resulting model compared to the linear coding. However, it has also more degrees of freedom. As I understand it, I cannot use the log likelihood test in this case, since both models are not nested.

What should I do?

> library(rms)
> 
> data(pbc)
> d <- pbc
> rm(pbc, pbcseq)
> d$status <- ifelse(d$status != 0, 1, 0)
> 
> dd = datadist(d)
> options(datadist='dd')
> 
> # linear model
> m1 <- cph(Surv(time, status) ~  albumin, data=d)
> anova(m1)
                Wald Statistics          Response: Surv(time, status) 

 Factor     Chi-Square d.f. P     
 albumin    73.51      1    <.0001
 TOTAL      73.51      1    <.0001
> 
> # rcs model
> m2 <- cph(Surv(time, status) ~  rcs(albumin, 4), data=d)
> anova(m2)
                Wald Statistics          Response: Surv(time, status) 

 Factor     Chi-Square d.f. P     
 albumin    82.80      3    <.0001
  Nonlinear  4.73      2    0.094 
 TOTAL      82.80      3    <.0001

UPDATE #1

I thought plotting both models would be a good way to decide whether a linear coding or a restricted cubic spline coding would be best. In this case (see below), I would think that the more complex coding is not better. However, the core of my question aimed to reinforce the eyeballing by a statistical test. But as I understand you correct, this is prone to over-fitting?

enter image description here

$\endgroup$
0
$\begingroup$

You can use AIC as a simple and intuitive way of penalizing for model complexity and see whether additional complexity is worth it in terms of incremental increase in explanatory power. This function is available in rms, and you would be looking for the model with the lowest AIC value.

However, note that you are comparing restricted cubic spline to a linear model, not the linear spline model. It may be justifiable in the exploratory data analyses mode, but aside from that, you should probably have an a priori justification of linear vs. non-linear model based on data distribution or expected response function. Otherwise, this fairly stringent penalty would often favor the simpler linear model.

$\endgroup$
2
  • $\begingroup$ I updated my question. $\endgroup$
    – Gurkenhals
    Jul 29 '15 at 11:32
  • $\begingroup$ It is hard/impossible to tell without seeing the data plotted alongside fit lines and/or AIC for each model, although your intuition is likely correct. That's right, adding extra terms increases the likelihood of overfitting, and AIC will account for that. $\endgroup$
    – katya
    Jul 29 '15 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.