2
$\begingroup$

I have a panel data set with right censoring. So I have individuals i with several time varying covariates Xit. For some individuals we observe event E (canceling their contract) and others we don't. The goal is to determine the effect of Xi on E. To consider the censoring structure of the data I wanted to use the Cox Proportional Hazard model. To control for time fixed factors, I wanted to demean the data (individual fixed effects). Here is my (simplified) approach in python using the lifelines package:

import pandas as pd
import dumpy as np
from lifelines import CoxTimeVaryingFitter

def demean_data(df, confounder, id_var):
    df[f'mean_{confounder}'] = df.groupby(id_var)[confounder].transform(np.mean)
    df[f'{confounder}_demeaned'] = df[confounder] - df[f'mean_{confounder}']
    return df.drop(columns=[f'mean_{confounder}', confounder])

df = demean_data(df, confounder, id_var)

ctv = CoxTimeVaryingFitter(penalizer=0.1)
ctv.fit(
   df, 
   id_col="id_var",
   event_col="event",
   start_col="start",
   stop_col="stop",
   show_progress=True
)
ctv.print_summary()

Now I found this article, which is confusing me, as it says that "fixed-effects Cox regression is not feasible when each individual experiences no more than one event". I do not understand, why fixed effects are unfeasible for Cox PH model.

$\endgroup$

1 Answer 1

1
$\begingroup$

A fixed effects regression model in this context eliminates the effects of time-fixed factors by de-meaning both the predictors and the outcome within each individual at each time point.

In a Cox survival model, the outcome is a time to event (potentially censored). Calculations are effectively done event time by event time, using the predictor values in place at each event time for all those at risk. There is no consideration of the history of the covariate values, just their current values and whether or not an individual had the event at that time.

If there is at most 1 event possible for an individual, then there is only one outcome value for each individual and effectively nothing to average over for the outcome in the context of a Cox model with its lack of memory about covariates. That poses a practical advantage with time-varying covariates, in that you don't need to deal with correlations within individuals if at most 1 event is possible per individual. As the R time-dependence vignette puts it at the end of Section 2:

One common question with this data setup is whether we need to worry about correlated data, since a given subject has multiple observations. The answer is no, we do not. The reason is that this representation is simply a programming trick. The likelihood equations at any time point use only one copy of any subject, the program picks out the correct row of data at each time.

The problem with that lack of correlation within individuals is that there is thus no way in a Cox regression to use a fixed-effects model to deal with fixed, unobserved outcome-associated factors.

The approach suggested in the reference you cite in this situation is to convert the Cox survival model to a series of binomial regressions over time. That provides a series of models for different time periods that provides a logical equivalent to the multiple observations over time in a standard fixed effects regression model.

$\endgroup$
3
  • $\begingroup$ thanks for your answer. One thing remains a bit unclear to me. Wouldn't it be sufficient to demean the covariates and not the dependent variable? I thought that would already get rid of the effect of time-fixed effects. What could be situations, where that could be harmful? $\endgroup$
    – TiTo
    Commented May 7 at 6:58
  • 1
    $\begingroup$ @TiTo fixed-effect regression accounts for an association between unknown fixed covariates and outcome. As the Wikipedia entry shows, when you subtract both the differences from average outcome and the differences from average covariate values, that automatically removes the part of the outcome that depends on the unknown covariates. If you only demean the covariates, you haven't done anything to adjust the outcome values for those unknown associations. You just end up changing the intercept. $\endgroup$
    – EdM
    Commented May 7 at 13:35
  • 1
    $\begingroup$ @TiTo this page covers when de-meaning/centering covariates can make sense. In survival models, which typically involve exponentiations, that can help with numerical stability problems in the calculation. Centering and scaling is done internally by the R coxph() function. If you use de-meaned but unscaled covariate values, in a Cox regression all that you do is change the baseline hazard; the relative hazards associated with the covariates are the same (within numerical precision). $\endgroup$
    – EdM
    Commented May 7 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.