0
$\begingroup$

I have a dataset of 100,000 composed by

  • the time during the study (time)
  • dead status (status); about 10% deaths
  • presence of 10 different traits as binary variables (trait1...traitX)

I'm using a cox-proportional hazard model to calculate the hazard ratio of the individuals for different traits (one at a time) and adjusted by sex (and other covariates like age) using R as follows:

coxph(Surv(time, status) ~ traitX + sex, data = dataset)

What I want to do next is to compare the individuals with traits with a hazard ratio higher than 1 against the ones without those traits to see if the mortality between these two groups is different (as is expected).

Here is my question. Do I need to split my initial dataset and use half to calculate the cox proportional model and half to calculate the mortality? If so, what is the best way to do it? Should I just randomly select 50,000 on each group?

Many thanks

$\endgroup$
  • 2
    $\begingroup$ This one-trait-at-a-time approach sounds like it could lead to some problems, as it doesn't allow for combined influences of multiple traits on survival. How many traits are you examining? How many deaths? Are you also adjusting for other covariates like age or measures of health status? Perhaps you could add a bit more about the nature of the hypothesis that you wish to test. On this site, it's best to add such additional information by editing the question itself, as comments aren't always read carefully and sometimes get deleted. $\endgroup$ – EdM Sep 18 '18 at 14:04
  • $\begingroup$ I'm examining 10 different traits, the number of deaths is around 10%. Most of the traits are unrelated between them but related to health and I'm also adjusting for age. $\endgroup$ – kakukeshi pani Sep 18 '18 at 14:15
  • $\begingroup$ I added the information from your comment into the question; please edit if I misunderstood somehow and changed the question erroneously. $\endgroup$ – EdM Sep 18 '18 at 16:25
2
$\begingroup$

With such a large dataset it can make sense to set aside a separate testing set, but your proposed one-trait-at-a-time approach doesn't take advantage of the scale of your data and might risk missing some important information about how the traits are related together to outcome.

With 100,000 cases having 10% deaths, you have 10,000 events to use in your analysis. The usual rule of thumb for evaluating predictors without overfitting a survival model is to allow at least 10-20 events per predictor. So you could in principle consider evaluating up to 500 predictors.

Just evaluating one predictor at a time is not a good approach for any type of regression model, as it ignores potential combined contributions to outcome. See this page for discussion in the context of linear regression with continuous variables. The issues, however, are similar with categorical predictors.

The regression coefficients in a multiple regression model including all predictors of interest then specify the relation of each predictor to outcome when all the other predictors are held constant, which is typically of more interest than single-predictor relations to outcome. A single-predictor relation to outcome can depend more on the distribution of its values among cases than on any intrinsic relation to outcome.

Besides covariates like age and sex, you have only 10 traits/predictors that you wish to examine. At the least you should use a multiple regression model that includes all of the traits and covariates at once. To look at how the 10 traits work together in more detail, you could consider evaluating all their 2-way interactions, of which there are only 45. This would show whether the effect of each single trait on outcome depends on the status of each of the other traits. A multiple regression model including all at once the other covariates, the 10 traits, and all the 2-way interactions among traits, and two-way interactions of the traits with covariates would only mean about 60 predictors.*

With respect to splitting the data, the danger is that limiting the number of cases used to build the model might cut down on the quality of the model, as it throws away information from the other cases. I don't have practical experience with such large data sets, but a 50/50 split as you propose seems like too large a test set. My sense is that using 80% for building the model and 20% for testing the model might give a better model with adequate testing.

Random sampling would be a wise choice, as it avoids any bias that might be associated with how the data were collected and formatted. If you know that there are some traits (or combinations of traits, if you choose to examine interactions) that occur infrequently, you might want to ensure that there is even representation between training and test sets by setting aside such cases and doing the random 80/20 (or other) split on such cases separately from the rest of the data.

However you split the data, though, avoid the one-trait-at-a-time approach and use a multiple regression instead.


*If you do choose to evaluate the large numbers of interactions among the binary traits, there is a danger that you might face a situation similar to the perfect separation seen in logistic regression. In that case you could use a penalized regression method like ridge regression, as noted on the page linked in this footnote.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.