4
$\begingroup$

I have a Cox model (estimated with coxph), built using "some" variables. There is an extra variable that I have not measured, so it is not in my dataset, but I do know the effect of that variable on cohort's similar to mine from literature (let's say the effect is well-established). If I want to "adjust" my model by including this effect, how do i do it?

My previous conviction was that this will lead to a biased model as we do not know about the interactions of this variable with the others in the cohort. But, I have seen some papers do something similar, such as Wishart 2012, where the author's adjusted their prognostic model for an extra variable that wasn't available in their cohort.

I am open to changing the base model (to a parametric or Bayesian), if that makes any difference.

Edit (for context): My goal is similar to Predict, to create a nomogram, which ideally I would like to adjust for an effect that is not in my dataset (large population level data, >10,000 observations, >500 events), but is well-established in literature.

$\endgroup$
3
  • 1
    $\begingroup$ What do you hope to accomplish by adding this unmeasured variable to your model? There was a particular rationale for the methods used in the Predict prognostic modeling, outlined here, to combine large-scale cancer registry data with results from smaller well-controlled clinical trials. What would your rationale be in your application? Please provide that information by editing the question, as comments are easy to overlook and can get deleted. $\endgroup$
    – EdM
    Jul 8, 2021 at 15:14
  • $\begingroup$ @EdM, thanks for the comment, edited for context. I am basically trying to build a nomogram similar to predict using population level data, while also wanting to adjust for an effect from literature. $\endgroup$
    – lvdp
    Jul 8, 2021 at 15:24
  • $\begingroup$ @EdM, BTW, your linked answer does provide some insight into how it is possible in theory, I am still conflicted about the assumptions and that we might be introducing some bias $\endgroup$
    – lvdp
    Jul 8, 2021 at 15:31

2 Answers 2

2
$\begingroup$

My goal is similar to Predict, to create a nomogram, which ideally I would like to adjust for an effect that is not in my dataset (large population level data, >10,000 observations, >500 events), but is well-established in literature.

My previous conviction was that this will lead to a biased model as we do not know about the interactions of this variable with the others in the cohort...

TL;DR

Overall, your caution is well advised. Correlations among variable values, interactions between variables with respect to outcome, and time dependencies can get in your way. It might be possible to do accomplish what you wish, if the studies in question adequately represent the underlying population of interest (as seems to be the case with Predict). There are, however, many ways for things to go wrong. The general issues can be gleaned pretty well from the Wishart et al. paper you cite, together with predictor-correlation issues noted in the answer by @wrstks.

What is to be gained by adding the "extra" variable?

From Wishart et al.:

The aim of this study was to incorporate the prognostic effect of HER2 status into Predict.

Based on your understanding of the subject matter, is the "extra" variable expected to improve performance enough to make this undertaking worth the investment?

Are the data sets providing the estimates for the "extra" variable comparable to yours?

From Wishart et al:

This model has been validated in case cohorts from British Columbia (Olivotto et al, 2005), the Netherlands (Mook et al, 2009) and the United Kingdom (Campbell et al, 2009).

What about your study and the studies on the "extra" variable you wish to draw from?

What are the correlations between the "extra" variable and those in your model?

From the answer by @wrstks

you would need to also consider published correlation values of that variable with your other variables...

I'm not so worried about parsimony in a predictive model as @wrstks seems to be (unless you're overfitting, more predictors generally help), but correlations do need to be considered. If there are significant associations among outcome-associated variables you would at least need to make sure that the source models took them into account appropriately. For example, from Wishart et al.:

We estimated the hazard ratio for HER2-positive disease compared with HER2-negative disease using a Cox proportional hazards model stratified by study and adjusted for size, grade and nodal status. Separate regression models were used for ER-positive and ER-negative cases.

Even then you will at least end up with problems in terms of confidence intervals for your estimates. Correlated outcome-associated variables have correlated errors among their individual coefficient estimates, so it's not clear how to combine the variances of coefficient estimates for correlated variables among different models that treat each individually. That's one reason why later validation of the combined model is important.

Are there interactions between the "extra" variable and those in your model in terms of relationships to outcome?

That's really tricky and needs serious consideration.

For example, the Predict model combined large-scale cancer registry data with much smaller but well controlled randomized trials on breast-cancer therapies to give a model incorporating both standard clinical variables and therapy, as outlined here. The registry data presumably covered the population of interest evenly, and the randomized trials evaluated the effects of therapy in a way that should have averaged out any confounding clinical variables.

Thus even if there were interactions between therapy and other variables in terms of outcome, the combined Predict model would at least have averaged the predictions properly over the population of interest. Can you say that about combining your data set with the studies that evaluated the "extra" variable?

It's possible that interactions with respect to outcome substantially affect the results of combining predictors, particularly in observational studies. In my work, for example, I have two very strong individual outcome predictors, not strongly correlated with each other but with an outcome-associated interaction such that if either one is a poor prognostic marker then the value of the other basically doesn't matter. Had their individual associations with outcome just been combined from separate studies, results would have been way off.

In addition to interactions among clinical variables, you need to consider their interactions with time with respect to outcome. Quoting from Wishart et al.:

the hazard ratio for HER2-positive disease decreases over time in women with ER-positive breast cancer, so the log hazard ratio was modelled to vary linearly with time. The effect of HER2 in women with ER-negative breast cancer is not time-dependent.

Might similar considerations of outcome associations in terms of interactions among variables or between variables and time apply to your situation?

If after that it still makes sense to do so, how do you combine hazard ratios among studies?

Cox models describe relative hazards that act around a baseline hazard. So if you want to combine models to provide survival estimates over time as Predict does, you have to consider not just the relative hazards but also the prevalence of the "extra" predictor to match a reasonable baseline hazard. Quoting extensively from Wishart et al. to provide a detailed example:

Predict was developed using a case cohort of breast cancer cases of unknown HER2 status, and so the underlying baseline hazard is representative of cases of average HER2 status. The HER2 hazard ratio estimates based on the BCAC data are for HER2-positive cases compared with HER2-negative cases, and so these were rescaled to give an average hazard ratio of unity using an estimated prevalence of HER2 of 9% in ER-positive cases and 25% in ER-negative cases. The [time-dependent] applied hazard ratios for ER-positive cases are shown in Table 3. A fixed hazard ratio of 0.93 for HER2–negative cases and 1.27 for HER2–positive cases compared with HER2 unknown cases was applied to the baseline hazard of the ER-negative model.

How do you propose to validate the combined model?

Unless you have a large data set that includes all the variables on which to validate your combined model it will be hard to convince anyone that the result is reliable. Quoting again from Wishart et al.:

We used the same case cohort from British Columbia, Canada, which we used to validate the original version of Predict. The data set has previously been described (Olivotto et al, 2005), but, in brief, includes data from 1653 patients with information on HER2 status out of a total of 3140 patients with stage I or II invasive breast cancer diagnosed in British Columbia, Canada, from 1989 to 1993...

Even then, I would worry about whether some bias had been introduced by omitting the nearly half of patients whose tumors lacked HER2 data. Was there some selection bias involved in terms of the decisions to look for HER2 that wouldn't let this validation set adequately represent the full population of interest?

$\endgroup$
2
$\begingroup$

The only thing you could do is simulate values for the variable you don't have, but you would need to also consider published correlation values of that variable with your other variables, which may not be available. If the variable you don't have is normally distributed, then you need to vary the correlation of the simulated quantiles (values of the variable you don't have) with your other inputs during simulations. Another complexity is that interaction is not correlation, it's a mean difference in the outcome over certain ranges of the interaction variables.

Consider the typical plot of SBP vs age, but two regression lines - one for males, one for females. A significant interaction between gender and age implies slope inequality - i.e., the fitted regression lines cross. That is, the predicted regression line for males has a slope that's significantly different from females.

Overall, this is a simulation problem. Everything you mentioned could be simulated, considering you know the mean and s.d. of each of the interaction variables.

Reason I raised the correlation issue is because if the variable you don't know correlates with one of your predictors, there will be redundancy of information and your model won't be parsimonious.

$\endgroup$
1
  • $\begingroup$ This new variable's correlation with other measured variables is of my main concern, as most often in literature we dont get published correlations. And assuming something which we dont know as the "truth" can lead to a more biased model than leaving that variable out. $\endgroup$
    – lvdp
    Jul 9, 2021 at 16:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.