How do I decide on keeping a variable with a p-value just above alpha?

Question

Goal: To find all critical variables that influence time to death so as to not record unneeded variables in future observations.

Call:
coxph(formula = Surv(time, DEATH_EVENT) ~ age + anaemia + creatinine_phosphokinase + 
    ejection_fraction + serum_creatinine + serum_sodium + hypertension, 
    data = HF)

                               coef  exp(coef)   se(coef)      z        p
age                       4.357e-02  1.045e+00  8.831e-03  4.934 8.05e-07
anaemia1                  4.460e-01  1.562e+00  2.150e-01  2.074   0.0380
creatinine_phosphokinase  2.101e-04  1.000e+00  9.825e-05  2.138   0.0325
ejection_fraction        -4.747e-02  9.536e-01  1.027e-02 -4.621 3.82e-06
serum_creatinine          3.139e-01  1.369e+00  6.895e-02  4.552 5.31e-06
serum_sodium             -4.569e-02  9.553e-01  2.336e-02 -1.956   0.0505
hypertensionPresent       4.965e-01  1.643e+00  2.137e-01  2.324   0.0201

Likelihood ratio test=80.58  on 7 df, p=1.048e-14
n= 299, number of events= 96 

This above selection of variables was derived from performing backward stepwise selection. 3 other variables were eliminated.

If you look at serum_sodium, you'll see a p-value of 0.0505 which is just over alpha = 0.05. I've never run into a situation in which the p-value is that close to alpha.

What steps are usually taken to decide on whether to include it or not?

Update

I performed a likelihood ratio test between the models and the null hypothesis, both models perform equally well, was not rejected. In the end, I dropped the variable but would appreciate responses had I not chosen this route.

Answer 1

Unless you need a parsimonious model with few predictors there is seldom a reason to omit an outcome-associated predictor from a model, provided that the model isn't overfit. Backward stepwise selection might be one of the least objectionable among the unreliable methods for automated variable selection, but that still doesn't often make it a good choice.

Since you did backward selection to arrive at the model you display, you should recognize that the p-values are already invalid. The calculation of those p-values assumes that the final model was built independently of seeing the outcomes. But that's not what you did. That makes any decision based upon a post-selection p-value even more unreliable. Look at the links suggested in a comment by @whuber for many more details.

Predictor elimination is particularly dangerous with Cox regression, which has an omitted-variable bias similar to what's found with binary regression. Even if an omitted outcome-associated predictor is uncorrelated with the included predictors, keeping it out of a Cox model can bias the estimates of the coefficients for the included predictors.

The best approach is to use your understanding of the subject matter to build a model as complex as reasonable while unlikely to overfit. Frank Harrell's course notes and book provide extensive guidance on how to proceed when you seem to have too many predictors for a limited amount of data. See Chapter 4 of each, in particular. Then, unless "parsimony is more important than accuracy," report the model results as is, without pruning the number of predictors.

In the question you say that you want to "find all critical variables that influence time to death so as to not record unneeded variables in future observations," which suggests that parsimony is important to you. But does it really make sense not to keep track of a common and easy-to-obtain predictor like serum sodium? With a Cox model, err on the side of inclusion.