0
$\begingroup$

I've been working on simulating survival experiments, and came across what seems to be an odd-looking artifact of Kaplan-Meier analysis which I have been struggling to explain in a concise manner mathematically. I've managed to convince myself that it's not an artifact of my simulation process itself. Here's an overview of what I'm doing:

  1. Generate survival times for a population of size n, randomly. Random values on (0,1) are drawn from a uniform distribution, and the survival times are then taken based on those values from a logistic distribution with a given median.
  2. Iterate over this population to simulate a survival experiment. Starting with the n members of the population, on each "day" (iteration) they are "scored" and the ones that are set to "die" on that day are recorded as such and not considered in future iterations.
  3. Continue iterating until the remaining population has 0 members.
  4. Repeat steps 1-3 10,000 times.
  5. Repeat steps 1-4 for a range of starting population sizes, all based on the same generating distribution.
  6. Fit each individual "experiment trial" with Kaplan-Meier (R survival package), and retrieve the median, mean, and 95% quantile survival estimates
  7. Summarize across the 10k "experiment trial" results for a given population size (i.e. s.d. of medians)

This is implemented in R, and I've run it both with and without ensuring reproducibility of random numbers, and with two different RNGs and saw the same patterns.

I see an odd oscillatory pattern in the summarized quantile survival estimates (median or 95% survival). I do not see such a pattern with the summaries of mean survival.

Here's an example, using 10k iterations per experiment "trial" on a range of simulated population sizes from 1 - 150 in increments of 5: Here's an example, using 10k iterations per experiment "trial" on a range of simulated population sizes from 1 - 150 in increments of 5

It gets more pronounced with looking at single-increment increases in population. Here's the same thing, but with 500 "experiment trials" (instead of 10k) and individual increments in population size from 5 - 300:

enter image description here

At that scale, the oscillation in the s.d. of the medians isn't really obvious, but both the mean and the s.d. of the 95% survival just looks wacky. The period of the repeating pattern in the mean of the 95% values across the increases in population seems to be 20. 20 is the median of the generating distribution I used, and I'm not sure if that's related.

I also see a pattern similar to the alternating one in the s.d. of the medians, in the medians of the medians when I show all the medians from each "experiment trial" as bars in a barplot. Since these are alternating, and the population size sequence is 5, 10, 15 etc, presumably that particular behavior is just because I'm going between even and odd population sizes, though I haven't explored that particular question in much depth. That alternating pattern also appears when I calculate s.d. or SEM or mean-squared error on the same data.

enter image description here

I don't have a reproducible example yet, but if anyone else out there has done this kind of large-scale simulation of survival and seen a similar pattern please let me know. I wasn't able to find anything relevant in the literature, based on the search terms I came up with at least.

$\endgroup$
0
$\begingroup$

These are pretty small oscillations relative to the magnitudes of the medians and 95th quantiles. For example, the range of medians in your first set of graphs is about $\pm 0.02$ around a value of 20.

t's hard to say for sure what's going on without seeing your code or a reproducible example. My guess is that this comes from the necessarily quantized nature of survival data and the odd/even population size issue. You don't seem to have censoring, so the median survival will be that of a particular observation with an odd number of cases, versus something between 2 observations with an even number of cases. The 95th quantile is even trickier, particularly when you have only a few dozen cases.

$\endgroup$
0
$\begingroup$

Am I understanding correctly that within an experiment you are sampling without replacement, and between experiments you are sampling with replacement from a finite population, except when a sampling unit expires? There is certainly merit for investigating such a scenario, but I think most of the theory behind survival analysis is based on sampling from an infinite population or sampling with replacement. At first this might seem unrealistic, but it can be justified for real-world applications by considering that as some people die or are simply no longer part of the target population of interest, new people become part of the target population. The assumption being that this "replenishment" of the population does not result in a time trend.

We could also envision the notion of repeated experiments as rewinding the hands of time for each experiment using a different sample of subjects. This would represent sampling without replacement for a single experiment, and with replacement between experiments. If the population is large enough the without replacement part is safely ignorable. This can then be viewed as sampling from an infinite population or with replacement from a finite population. Let me know if I have made any mistakes above.

Sometimes unusual or correlated results can occur in simulated data depending on how the numbers a generated. For instance, in SAS it is better to simulate all of the replications in a single data set rather than writing a macro that creates a new data set for every replication. The former ensures the random number generation process is independent while the latter does not.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.