# Do you have to remove perfectly collinear independent variables prior to Cox regression?

Suppose you have independent variables that change only with each new time step (and possibly others that change freely):

id   time   X_1   X_2   X_3     …   event
01      1   a_1   b_1   c_1     …       0
02      1   a_1   b_1   c_1     …       0
⁝       ⁝     ⁝     ⁝      ⁝      …       ⁝
01      2   a_2   b_2   c_2     …       ⁝
02      2   a_2   b_2   c_2     …       ⁝
⁝       ⁝     ⁝     ⁝      ⁝      …       ⁝
01      n   a_n   b_n   c_n     …       ⁝
02      n   a_n   b_n   c_n     …       ⁝


At time=1, all values of X_1 are equal to a_1 (which may or may not equal a_2, a_3, ...) for all ids. Similarly, at time=i, all values of X_1 are equal to a_i for all ids, and so on. Independent variables X_2 and X_3 also change only with each new time step. Other variables might change freely, but they are not important to the question. Are independent variables X_1, X_2, X_3 perfectly collinear? And should they be removed prior to Cox regression due to collinearity or otherwise (zero variance per time step)?

To be clear, all individuals have exactly the same value for any one of X_1, X_2, X_3 at any given time point.

• Is this any different from what would happen with age if you updated it as a predictor at each time point? Multicollinearity has to do with correlations among predictor variables, not within a single predictor. Are you also saying that the values of X_2 or X_3 are changing in parallel with the values of X_1 in a way that introduces linear dependence among them? Please provide that information by editing the question, as comments are easy to overlook and can get deleted.
– EdM
Jan 12, 2022 at 18:39
• "Are you also saying that the values of X_2 or X_3 are changing in parallel with the values of X_1 in a way that introduces linear dependence among them?" The values of X_2 and X_3 also change only with each new time step. I am not sure if it introduces linear dependence or not. For example, age and year have a linear dependence, but what about year and annual_inflation_rate, or a less obvious example, annual_inflation_rate and annual_interest_rate. Jan 12, 2022 at 18:53
• No these variables are not perfectly collinear. Supposing, say, you had height, weight, and age. Obviously they all change together, but they don't perfectly predict one another. Of course, the Cox model is in many ways like a categorical analysis, such as logistic regression, so that certain expressions of the model matrix that might converge with a linear regression will fail in a Cox or logistic model due to separation. Jan 12, 2022 at 19:18
• @AdamO So they are not perfectly collinear, but should they be removed prior to Cox regression due to zero variance per time step? How would the regression model know which variable is affecting the hazard? Jan 13, 2022 at 15:42
• @user572780 you say 0 variance but you mean 0 variance within a cluster id if your data example is setup correctly. Consider the whole sample. If the X_i is completely constant within each value of id, that's fine as long as there's variability between each id. It amounts to adjusting for a between-cluster effect in a panel regression. Jan 13, 2022 at 16:40

Perfect multicollinearity among predictors should be dealt with in any regression, Cox or otherwise. Some software might just refuse to fit such data. Some functions are smart enough to find and remove enough predictors to ensure linear independence (perhaps silently), but you don't want to count on that and you presumably would prefer to make such a choice consciously yourself.

Multicollinearity doesn't arise from associations of values within single predictors like your X_1. It has to do with linear associations among predictors. There's no way to say in general whether the type of data structure you describe will lead to perfect multicollinearity among X_1, X_2 and X_3.

The problem in your scenario is that you might not gain any information from including covariates that have "zero variance" among members of the risk set at all event times in a Cox model. If all individuals at risk at each event time have identical values of a particular covariate $$X_j(s)$$, and that is true for all of the event times $$s$$, then you can't estimate a hazard for that covariate unless it has interactions with other covariates.

The vector of coefficient estimates $$\hat \beta$$ in a Cox model comes from solving the partial-likelihood vector score function $$U$$ as a function of the covariate-coefficient vector $$\beta$$ for $$U(\hat \beta)=0$$. Following Therneau and Grambsch on page 40, with p covariates and n individuals, the $$p \times 1$$ score vector is:

$$U(\beta) = \sum_{i=1}^n \int_0^{\infty} [X_i(s) - \bar x(\beta,s)]dN_i(s),$$

where $$\bar x(\beta,s)$$ is a risk-weighted mean of the $$X$$ values over the risk set at time $$s$$ and $$dN_i(s)$$ is 1 at the event time for individual $$i$$ and 0 otherwise.

If the entire risk set at an event time $$s$$ has identical values of $$X_j$$ and that holds true for all values of $$s$$, then the $$j^{th}$$ component of the score vector is necessarily 0 regardless of the coefficient values $$\beta$$. There is thus no unique solution for $$\hat \beta_j$$. If $$X_j$$ is involved in interaction terms with other covariates, however, then estimates of those interaction coefficients could be possible.

This difficulty with modeling such covariates in a Cox model might be overcome in a parametric survival model. With a parametric model, covariates can contribute to the entire likelihood function over time, as summarized on this page, and thus allow for estimation of their associated coefficients if they vary over time. Unless they follow defined functions of time, however, time-varying covariates for an individual are modeled over a set of time spans, each of which is left truncated and is either right-censored or ends with an event. If you choose an accelerated failure time parametric model, handling left truncation requires some care as explained in this vignette of the R eha package.

For reference of others reading this thread, know that time-varying predictors in a Cox model are typically formatted in a different way than shown, with one data row for each combination of individual and time period during which all covariate values are constant. That's sometimes called the counting-process format. Each row specifies the start time (left truncated) and end time (censoring/event) of each such time period, along with the covariate values and an event/censoring indicator. See the R time-dependence vignette for details.

• I have changed the format of the data. The variables are not perfectly collinear. What about variables with zero variance per time step (the case here)? Should they be removed? Jan 12, 2022 at 19:56
• @user572780 it's hard to say without more details of the data. For Cox models it's generally best to incorporate as many predictors as you can without overfitting, to minimize omitted-variable bias. Even if all of the individuals have the same value of a covariate at any particular time, the changes in estimated hazard as that covariate value changes over time will provide some information. You might, however, have problems trying to make predictions from new sets of time-varying covariate values. See this thread for example.
– EdM
Jan 12, 2022 at 20:17
• I have difficulty understanding why I should include variables with zero variance per time step. How would the regression model know which variable is affecting the hazard? I was thinking of combining all of the variables with zero variance into a new term. Is that a reasonable thing to do? Jan 13, 2022 at 15:40
• @user572780 I maybe hadn't completely internalized that for any one of those predictors at any given time point, all individuals have exactly the same value. So you are correct--if each is modeled independently of other predictors. If some of those predictors are involved in interactions with other predictors that don't have zero variance, however, you would need to include them to model those interactions. Otherwise, I don't see that combining all of the zero-variance predictors gains you anything at all--you still have a combined predictor with zero variance at any time point.
– EdM
Jan 13, 2022 at 16:32
• Okay, that clears it up. I was thinking that there is still information to gain by including a single predictor with zero variance at any time step. For example, if you keep the year predictor, which has zero variance at any given time step. Then, you might notice that when year=2021, there was an increase in loan defaults. And you might interpret that as an increase in the hazard due to year=2021 (or annual_inflation_rate or annual_interest_rate or any other predictor with zero variance at any time step). Jan 13, 2022 at 18:08