2
$\begingroup$

I'm using the lifelines Python package to learn Cox Proportional Hazard (CPH) model. What I need now is to feed it new examples and generate the predicted hazard rate (the probability of the event occuring at time t, given that the person has survived up to time t). (BTW, sorry if I'm getting my terms mixed up between hazard rate and hazard function, but either way, I need a probability of the event happening from the CPH model).

I tried the CoxPHFitter.predict_partial_hazard(), but the values coming back are exceeding the range of [0, 1]. For example, I'm seeing values as high as 11.028 or 60,326.305. This leads me to believe that this value being returned is not the hazard rate (of course, the code documents that this is just the partial hazard). As I stated before, the hazard rate is a probability, and obviously, this value exceeds the bounds of [0, 1]. Anyone know if I get the the hazard rate using this Python package?

Additionally, if there's anyway to do this in R, I'm willing to try that too.

There is a similar question here, but it focuses on R and also the accepted answer did not even answer the question, IMO.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
6
$\begingroup$

Author of the lifelines library here. Here is the details as of v0.14.4.

To help, let's review the from the the CoxPH model in lifelines:

$$ \lambda(t | x) = \overbrace{b_0(t)}^{\text{baseline}}\underbrace{\exp \overbrace{\left(\sum_{i=1}^n \beta_i x_i \right)}^{\text{log-partial hazard}}}_ {\text{partial hazard}} $$

The partial hazard in the CoxPH model is the $\exp(...)$ part, so I don't expect it to be between 0 and 1.

As I stated before, the hazard rate is a probability

This is the case in some discrete model (Nelson Aalen model for example), but not true in the Cox model.

What I need now is to feed it new examples and generate the predicted hazard rate (the probability of the event occuring at time t, given that the person has survived up to time t).

To do this, here is what I am thinking. Each individual will have a survival curve sf = predict_survival_function(individual), and we can produce the conditional survival function for an individual and compute the delta probability between two points to answer "what is the probability the event occurs in this time period?".

Improve this answer
$\endgroup$
4
  • $\begingroup$ Could you please clarify? A data frame is passed back; are the values under the 0 column the survival probabilities and the row indexes the time intervals (e.g. years)? If I had the survival passed back for an individual who's in his 1st year, could I filter for the closest points in time e.g. indices = [index for index in scurve.index if 0.95 <= index <= 1.01] and take the difference of the last 2 survival probabilities as the probability of the event in this time period? $\endgroup$
    – Jane Wayne
    Commented Jul 12, 2017 at 21:27
  • $\begingroup$ And also, if I needed to get his survival probability 2 years out (e.g. 3 years), I would filter out again indices = [index for index in scurve.index if 2.95 <= index <= 3.01] and take the difference between the last 2 probabilities. What I am trying to produce is, what is the current probability of the event now, and 1, 2, 3, etc... years out? Does these approaches make sense? $\endgroup$
    – Jane Wayne
    Commented Jul 12, 2017 at 21:31
  • $\begingroup$ Now that I'm thinking about it and doing some code, if I wanted to know the probability of failure at year 1, 2, 3, etc... can't I just find the closest row index value, take its corresponding survival probability, and do 1 - survival_probability to get event probability? $\endgroup$
    – Jane Wayne
    Commented Jul 12, 2017 at 22:00
  • $\begingroup$ Your comments are all reasonable. For your latest comment, yes, but now your interpretation of " if I wanted to know the probability of failure at year 1, 2, 3..." is "the probability of the event before year 1, 2, 3...". It's not necessarily the probability of event on year 1, 2, 3. $\endgroup$
    – Cam.Davidson.Pilon
    Commented Jul 13, 2017 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.