4
$\begingroup$

Singer and Willett (2003) write the following about estimating the standard errors of estimated survival probabilities within the context of discrete time event history models (e.g. logit hazard models):

Estimating the standard error of a survival probability is a more difficult task than estimating the standard error of its associated hazard probability. This is because, unlike hazard, which is estimated as the fraction of the risk set that experience the target event in any given period, the survival probability is estimated as a product of (1 – hazard) for this and all previous time periods. Estimating the standard error of an estimate that is itself the product of several estimates is a difficult statistical task. Indeed, it is so difficult that statisticians rarely recommend that you estimate the standard error of the survival probabilities directly..." [Emphasis added]

I am interested in how I can better understand why they made the bolded assertion (specifically, because they are correct, I and some collaborating statisticians are having a very difficult time producing a reliable estimate of the variance of the survivor function). Why is this kind of estimate especially difficult?

References
Singer, J. D. and Willett, J. B. (2003) Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford University Press.

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ What are their definitions of hazard and survival? I don't see how to make survival a product of hazard. I'd have expected to see $S(t) = \exp(-\int_0^{t} \lambda(u)\,du)$. Certainly $S$ can be written as a product ... but a product of single-period survival probabilities, or a product of $1- p_i$ where $p_i$ are single-period failure probabilities. Single period failure probabilities are not normally referred to as 'hazard'. $\endgroup$
    – Glen_b
    Commented Apr 9, 2015 at 5:00
  • $\begingroup$ @Glen_b This is discrete-time event history analysis on a person-period data structure (logit hazard, complimentary log-log hazard, and probit hazard models). Check out the link to my other question... I go into more detail there. $S_{t} = \prod_{i=1}^{t}{(1-h_{i})}$, where $t$ is the number of discrete time periods elapsed before an event happens, and $h_{t}$ is the probability of event at time $t$ conditional on the event not having yet occurred. $t$ can also be thought of as the number of failures before event. $\endgroup$
    – Alexis
    Commented Apr 9, 2015 at 5:47
  • 2
    $\begingroup$ okay, thanks, looks like differing conventions in different areas. It's odd that they call this hazard; the area I'm used to working in they're very careful to call those things probabilities, and distinguish probabilities from hazards. I'm going to suggest we leave this discussion here, since it may be more than just me that finds the terminology a surprise. $\endgroup$
    – Glen_b
    Commented Apr 9, 2015 at 6:35
  • 2
    $\begingroup$ Linear combinations of random variables (like sums) are rather unusual in that they have some very simple properties when combined with expectation and variance operators. Things that don't work linearly (like products instead of sums) ... in general can't rely on such nice properties, so are usually hard to work with. There are various tricks and approximations that are sometimes helpful. $\endgroup$
    – Glen_b
    Commented Apr 9, 2015 at 6:41
  • 1
    $\begingroup$ Well, if you have independence, expectation of the product is easy, and variance of the product is not all that much harder. If you have a very large number of terms in the product (and certain conditions hold), you can sometimes argue for approximate normality on the log-scale. Sometimes Taylor expansions can be used. $\endgroup$
    – Glen_b
    Commented Apr 9, 2015 at 15:17

1 Answer 1

Reset to default
4
$\begingroup$

Consider the variances of a random variable X: $$Var[X]=\sum_i (x_i-\mu_x)^2 Pr(x_i)$$ here $Pr(x)$ is the probability of value $x$.

Now, consider a variance of the product of variables X and Y: $$Var[XY]=\sum_i\sum_j x_iy_j Pr(x_i,y_j) $$

So, the difficulty comes from the necessity to know the joint probability $Pr(x,y)$. Imagine that in the case of the survival probability it's not just two variables, it's the product of hazards in each period. You'll end up with a joint probability function with as many dimensions as periods. If you do the continuous time survival (hazard) analysis it gets even crazier.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.