2
$\begingroup$

I have some data where I follow some people for 365 days. I have a response variable that is either 1 or 0. The response variable is an event that may or may not happen between day 1 and day 365. If the event happens (1), I know at what day it happens. I've been doing logistic regression with success.

Now the data has changed, and some of the customers suddenly have varying exposure (less than 365 days). For example, some people we'd only follow for 180 days. I still know if an event happens before the 180 days (of exposure), but if any event happens after the 180 days is unknown.

My manager says that I can still do logistic regression, I just have to take into account the varying exposure, that is, include the exposure as an explanatory variable in the model. This just feels wrong to me, and I say that this is what cox regression/survival analysis is made for, but I can't really argue for why it is wrong to do it the suggested way.

Is my manager wrong, and if so: could somebody please explain why?

$\endgroup$
2
  • $\begingroup$ Please edit the question to say more about the nature of the response variable. For example, can it switch back and forth between 0 and 1 for the same individual over time, or is it always at 1 once it reaches that value? If the latter, do you know the time of the occurrence of the switch from 0 to 1, or just that it happened some time within the 365 days of observation? $\endgroup$
    – EdM
    Jan 16 at 14:24
  • $\begingroup$ @EdM done! Can't switch back and forth. If an event (1) happens, it is always 1. Imagine this is somebody dying or defaulting. And yes, we know the time of occurence. $\endgroup$
    – Erosennin
    Jan 16 at 20:37

1 Answer 1

3
$\begingroup$

Why is Logistic Regression Wrong?

If you had observed all subjects for the same amount of time, logistic regression would be fine. The censoring -- not knowing what happens after 180 days -- is the problem.

Think of this in an extreme case. Suppose we're interested in examining the risk of death. However, our sample consists mostly of children (who do not typically die young, and often go on to live into adulthood). The short observation time of the children means they will likely contribute an excess of 0's to our estimate of the risk of death. Hence, that estimate is inappropriate to apply to the entire population; the children simply have not enough time to accrue risk of death by ... much of anything. Adults, by contrast, have had more opportunities to die from crossing roads, smoking cigarettes, having plaque accumulate in their arteries, and so on and so on. Observing all subjects for the same amount of time means they have had (assuming all else equal) the same time to accumulate the same risk.

How Can We Analyze Such Data?

You could use a cox model, but you could also do a Poisson regression with robust covariance estimation and use exposure time as an offset, as is explained here. The {survival} R library has a function called pyears which calculates the length of exposure. The docs for pyears show an example of how to use the exposure time as an offset, but they do not show how to estimate robust covariance in that example.

Let's see how to do this with an example. First, let's simulate some data. Assume we've randomly assigned patients to two groups. Group membership is indicated by trt.

N <- 1000
trt <- rbinom(N, 1, 0.5)
raw_tm <- rexp(N, 1/(100 - 50*trt))
cens_tm <- 365
tm <- ceiling(pmin(raw_tm, cens_tm))
event <- 1.0*(raw_tm < cens_tm)

d <- data.frame(tm, event, trt)

Now, let's use pyears to count the outcomes in each group and the total person years contributed.

library(survival)
p <- pyears(Surv(tm/365, event)~trt, data=d, scale=1, data.frame=T)

p$data
> p$data
  trt    pyears   n event
1   0 145.32877 519   503
2   1  67.09041 481   481

There were 503 events in the 145 person years contributed by the trtr=0 group. That's a rate of 503/145.33 ~ 3.46 events per person year. This means if I followed 10 people for a year I would expect 10 people x 1 year x 3.46 = 34.6 events. Same if I had followed 5 people for 2 years, or 20 people for 6 months.

Now, onto the regression

fit <- glm(event ~ trt + offset(log(pyears)),
           data=p$data,
           family = poisson)


coeftest(fit, vcov. = sandwich)

z test of coefficients:

              Estimate Std. Error    z value  Pr(>|z|)    
(Intercept) 1.2416e+00 2.5167e-32 4.9334e+31 < 2.2e-16 ***
trt         7.2823e-01 8.2724e-16 8.8031e+14 < 2.2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

The base rate of events in the trt=0 per 1 person year is exp(1.24) ~ 3.46 -- which is exactly what we saw. We can also see that the rate of events per 1 person year in the trt=1 group is exp(0.73) = 2.07 times as large as compared to the trt=0 group. That also makes sense from the pyear tabulation we saw above.

So far as the 180 day cliff is concerned, your data are censored at 180 days. That is still informative, because we know that those individuals do not get the outcome within 180 days. Their person years and 0 events still count towards the regression.

$\endgroup$
10
  • $\begingroup$ Thanks for your answer! Regarding your "Why is Logistic Regression Wrong?" section, I don't understand why this explains to me what it is wrong to use logistic regression + account for the exposure using an explanatory variable. My manager would say just use age of the people as an explanatory variable (or offset). What is wrong with that? And thank you very much for the thorough answer in "How Can We Analyze Such Data?", much appreciated <3 $\endgroup$
    – Erosennin
    Jan 17 at 7:08
  • $\begingroup$ @Erosennin even if adjusting for exposure time in a logistic regression were right, you still have the problem of censoring. Censored observations are not actually 0s -- had you observed them the entire 365 days, some of them might actually get the outcome. By using a logistic regression, you're not estimating the risk correctly. Simulate this yourself and see how risk predictions from a logistic regression differ from the true survival probability. $\endgroup$ Jan 17 at 15:17
  • $\begingroup$ To play devils advocate: I could argue they are actually 0s at the point of censoring, why would that be wrong? I know some of them would get the outcome if I followed them longer, but so would the poeple that I only observe for 365. Follow anything longer and the risk increases. But I would account for that using the exposure. $\endgroup$
    – Erosennin
    Jan 17 at 16:21
  • $\begingroup$ I could simulate, but I would like to help my intuition understand it better, so I can explain it to my manager verbally :) $\endgroup$
    – Erosennin
    Jan 17 at 16:56
  • $\begingroup$ @Erosennin OK, so it turns out this CAN be done, however you need to let the term for the length of exposure be sufficiently flexible. This paper by Efron demonstrates how $\endgroup$ Jan 19 at 22:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.