Is a pairwise Cox regression approach valid for small samples?

Question

Firstly, some background on the dataset:

I am performing survival analysis on a 28-event dataset. We found a genetic marker that predicts survival on a drug. Examining the data, the proportional hazards assumption appears to be met. I want to perform Cox regression with the goal of identifying potential confounders.

The issue is that with a 28-event dataset, I can't really fit more than 2 independent variables at a time to the data without risking overfitting. My approach has been to fit each potential confounder in turn together with my variable of interest (the genetic marker) and observe the impact on the regression.

My question is whether this is a valid approach for determining whether these other independent variables are confounders or not. Is there anything fundamentally wrong with this "pairwise" approach - aside from not being able to catch confounders which may have an impact on the regression together but not apart?

Answer 1

As Frank Harrell puts it on page 2 of Regression Modeling Strategies, "Testing too many hypotheses is related to fitting too many predictors in a regression model"; and "If the sample size is insufficient for modeling it is often insufficient for tests or estimation."

If you were to do regressions with multiple selections of a second variable to combine with your marker, you would at least have to consider the dangers inherent in multiple testing. Note that this approach could lead to both Type I and Type II errors in terms of evaluating the prognostic importance of your marker.

There is a suggestion from simulation studies that the rule-of-thumb of 10 events per predictor can be somewhat relaxed when adjusting for confounders, perhaps down to as low as 5 events per predictor, but one should proceed cautiously and use procedures like bootstrapping to try to evaluate potential over-fitting and bias.

With only 28 events, you are evidently still in the exploratory stage of this study. From that perspective, playing with your data in the way you suggest may help point the way to better design of later studies to evaluate the marker rigorously. But keep in mind the dangers inherent in trying to glean too much information from too few data points.

One further note: as you proceed with your study, think closely about what you mean by "confounder." If you are simply adjusting for other variables that might affect outcome, then a simple multi-variable model makes sense. If, however, "confounders" are thought to affect the relation of your marker to outcome, then you need to incorporate interaction terms in your model, and you will need even more events.

Answer 2

My advice is to include the most important known prognosis factors as confounding variables, in the model. This could be a problem with a small sample size, but is the way to make sure that you consider all the relationships between the genetic variable and the prognosis factors.