2
$\begingroup$

I'm interested in modeling the Hazard Rate $\lambda$ from a Survival dataset so I can calculate the Cumulative Distribution $F(t)=1-e^{-\lambda t}$ but I'm not sure how to go about computing $\lambda$

I know that in the continuous case the Hazard Rate is defined as $$\lim_{\Delta t \to 0} \frac{P[t<T<t+\Delta t | T>t]}{\Delta t}$$ But I'm not sure how to implement this practically. I've seen so many suggestions online from Cox PH to B-Splines and Bayesian methods but without much elaboration and which only got me even more lost. Hope someone can clarify this for me.

Thanks.

EDIT:
Here's a sample of the data: survdata

It contains 50 observations with 4 attributes. First one is status of Checking Account that goes from 1 to 4 indicating different intervals of deposit amounts, second is the number of days Survived by that individual before default, third is the amount of credit given to the firm, and fourth is the status of default with 1 indicating a default event for that individual and 0 means no default.

$\endgroup$
  • 2
    $\begingroup$ Do you think your survival time follows exponential distribution? $\endgroup$ – user158565 Aug 5 at 14:57
  • 3
    $\begingroup$ Could you please say more about the nature of your data and your goals in modeling the hazard rate? $F(t)=1-e^{-\lambda t}$ only holds when the instantaneous hazard is constant in time. It's also not clear whether you want your model to account for covariates that might be related to survival. Depending on your data and the purpose of your modeling, you could do parametric modeling of hazard over time (e.g., Weibull models), or extract an empirical baseline hazard from a Cox proportional hazards model, while accounting for covariates. $\endgroup$ – EdM Aug 5 at 15:35
  • $\begingroup$ It's part of a postgrad project where I'm trying to estimate Probabilities of Default for a dataset composed of a group of firms with a fraction that defaults at different times. My goal in modeling the hazard rate is to use it to compute the Cumulative Function $F(t)$ using that formula but I see now from your comment that I'm making an assumption that isn't necessarily true (constant hazard). I'm new to survival analysis and I'm lost in the midst of all the literature. How do you suggest I should go about it? If you don't have the time to explain it's fine a source is also welcomed. Thanks. $\endgroup$ – Metrician Aug 5 at 20:05
  • $\begingroup$ After doing some more digging around I see 3 ways that this can be done but I don't know if there would be some complications hope someone can help: 1) Use Kaplan-Meier Estimate, use $S(t$) to derive $H(t)$ and then differentiate it to get $h(t)$ 2) Use Nelson-Aalen Estimator to get $H(t)$ and differentiate it to estimate $h(t)$ 3) Fit a distribution (Exponential, Weibull etc..) over Kaplan-Meier estimated $S(t)$ and compute $h(t$) analytically. Are any of these approaches feasible ? As for "extracting empirical baseline hazard from Cox PH I'm not sure how exactly I should go about that $\endgroup$ – Metrician Aug 5 at 23:02
1
$\begingroup$

The Kaplan-Meier estimator for the survival function and the Nelson-Aalen estimator for the cumulative hazard have step changes at event times, so trying to differentiate with respect to time (as you suggested in a comment) to get the instantaneous hazard won't work. The derivative will be 0 between event times and infinite at event times.

It's not clear why you need to know the instantaneous hazard as a function of time if your interest is in a cumulative hazard like your $F(t)$. If there are no covariates to account for, then it seems that what you want is the Nelson-Aalen estimate itself, which describes cumulative hazard for the data set at hand while taking censoring into account. If you wish to control for covariates, then you could fit, say, a Cox proportional hazards model with a statistical software package and use the associated prediction functions to generate plots that illustrate how those covariates affect survival. The survival functions will still have step changes at event times, however.

If you want a smooth approximation to the data then you should fit some parametric survival model to the data, again potentially including covariates. A Weibull model often works well and has more flexibility than the constant-hazard model implicit in your equation for $F(t)$; that constant-hazard model is one particular example of the more general Weibull model. That's one of several parametric possibilities provided by standard statistical software packages. With such a smooth parametric approximation to the survival curve then you can differentiate with respect to time and get instantaneous hazards if you still need that.

In response to information added to question:

Your data are certainly in a form suitable for standard survival analysis (Kaplan-Meier, Cox models, parametric survival regression): a wide enough range of times (from 6 to 60) that it doesn't seem necessary to use discrete-time analysis, some covariates to consider, and annotations of event (default) versus censored (no default at last observation time), with a separate row containing these values for each individual. For predictions about survival based on your data, you don't have to re-invent the methods yourself (as the start of your question might be taken to mean). You can just use the modeling and prediction facilities provided by any standard statistical software system that handles survival data, which incorporate all the issues that you raise (and more).

For example, in the basic R survival package there is a predict() function that can handle either semi-parametric Cox proportional hazards models (coxph() with its implicit baseline hazards) or fully parametric models (survreg() with 6 built-in choices of assumed distributions, including Weibull). You fit a model to the data you have, then provide the model along with the covariate values for any new case for which you wish a survival prediction. (It's good practice also to request the standard errors for the prediction.)

The above assumes, however, that DaysSurvived=0 in your data represents some appropriate reference time that applies equally for all cases, for example the date on which the loan was made to that individual, and that the covariate values are those that held for each individual at the corresponding DaysSurvived=0. Otherwise a more complicated analysis might be necessary.

$\endgroup$
  • 1
    $\begingroup$ @Metrician discrete time survival analysis can be somewhat different from what you get with Cox models or continuous time parametric models. I’m traveling and won’t be able to get back to this for a couple of days. In the meantime, please edit your question to provide a sample of the type of data you are dealing with, in particular whether there is a time=0 defined similarly in some way for all firms. Please edit the question, not just add the information to a comment as comments can get lost. $\endgroup$ – EdM Aug 21 at 20:26
  • 1
    $\begingroup$ @Metrician added a bit to the answer, in response. $\endgroup$ – EdM Aug 27 at 21:39
  • 1
    $\begingroup$ @Metrician a quick plot of your example data suggest that all default by t=60, so what you would get is the probability of default before a specified time, given a set of covariates. For example, call the predict.coxph function with a Cox model based on your data, a set of covariates and time of interest in the newdata argument, and specify type="expected". As the manual page for predict.coxph says: "The survival probability for a subject is equal to exp(-expected)." $\endgroup$ – EdM Aug 28 at 14:28
  • 1
    $\begingroup$ @Metrician that gives the probability of defaulting before the time specified when you ask for the prediction. As I noted, your example cases all seem to default by t=60. If you want to distinguish "eventually defaults" from "never defaults" you will need to include cases that don't default and specify the time that is late enough such that later times can be deemed "never." $\endgroup$ – EdM Aug 29 at 14:12
  • 1
    $\begingroup$ @Metrician what you wrote in your last comment makes sense [flipped around, as what your wrote will necessarily be negative, so with SP = survival probability you want SP(t-1) - SP(t) for probability at time = t] if you are dealing with discrete time points. With a continuous time axis, you typically care about SP up to a particular time t or between two specific time points. Your formula in the last comment (as I rewrote it) works exactly in that case and agrees with your formula in the comment before that: SP(0) = 1 by definition, so prob. of default before time t = 1-SP(t). $\endgroup$ – EdM Sep 1 at 15:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.