How to log-transform the confidence interval for the survival function?

Question

I want to improve my understanding of confidence intervals for product-limit estimates of the survival function. I'm using the book by Klein and Moeschberger as a reference.

The product-limit estimator is defined as

$\hat{S}(t)=\prod_{t_i\leq t}(1-\frac{d_i}{Y_i})$

where $(t_i)_i$ are the observed, right-censored times, $d_i$ is the number of deaths at time $t_i$, and $Y_i$ is the number of patients in the study up to time $t_i$. Its variance can be estimated by

$\hat{V}(\hat{S}(t))=\hat{S}(t)^2\sum_{t_i \leq t}\frac{d_i}{Y_i(Y_i - d_i)}$.

For the confidence intervals, define

$\sigma_S^2(t) = \frac{\hat{V}(\hat{S}(t))}{\hat{S}(t)^2}$

and let $Z_a$ denote the $a$ percentile of a standard normal.

Then the linear confidence interval with level $1-\alpha$ is

$\hat{S}(t_0) \pm Z_{1-\alpha/2}\sigma_S(t_0)\hat{S}(t_0)$

The log-transformed confidence interval is

$[\hat{S}(t_0)^{\frac{1}{\theta}}, \hat{S}(t_0)^{\theta}]$ with $\theta=\exp(\frac{Z_{1-\alpha/2}\sigma_S(t_0)}{\ln(\hat{S}(t_o)})$

which is based on log-transforming the cumulative hazard function, or transforming the survival function via $\ln(-\ln(x))$.

My understanding of this transformation is fuzzy. Is it just simple algebra? Do I have to know anything about asymptotic statistics?

So my question is, how is the log-transformed confidence interval obtained?

Answer 1

There are a few things beyond simple algebra that go into the derivation:

The linear confidence interval you list relies on the fact that the sampling distribution for the estimator $\hat{S}(t)$ is asymptotically normal (so your confidence interval really should be produced from a data set of reasonable size). This, in turn, can be shown by showing that $\hat{S}(t)$ is a maximum likelihood estimator, and it is a general fact that MLEs are asymptotically normally distributed.

Then you must know the invariance property of MLEs, which tells you that $ln(\hat{S}(x))$ is also an MLE. It is therefore asymptotically normally distributed, and its expected value is $ln(S(x))$. Thus,

$$ \frac{\ln (\hat{S}(x))}{\ln (S(x))} $$

is approximately normal with mean 1.

Now, as you say, we take another logarithm and use the invariance property again to determine that

$$ \ln \left( \frac{\ln (\hat{S}(x))}{\ln (S(x))} \right) $$

is approximately normal with mean 0.

The construction of the log-transformed confidence interval now follows by simple algebra if you can determine the variance of the above estimator. This is probably nigh on impossible to do exactly, but once more we can get an approximate value, this time using the delta-method.