3
$\begingroup$

I am interested in learning about how decisions (e.g. prioritization of treatment) can be made in the real world based on Survival Curves (e.g. Kaplan-Meier).

To illustrate my example, I simulated a dataset of survival times for two cohorts ("Blue" : Men, "Pink": Women) using the R programming language. In this dataset (e.g. Years Survived After Liver Surgery) , I purposefully try to show that Men initially have better survival probabilities compared to Women - but as time goes on, Women have better survival probabilities (For the sake of simplicity, I assume that there is no censoring):

library(survival)
library(ranger)
library(ggplot2)
library(dplyr)
library(ggfortify)

#create 4 mini datasets data
times = rnorm(100, 20, 1)
group = 1

data_1 = data.frame(times, group)

times = rnorm(100, 10, 1)
group = 0

data_2 = data.frame(times, group)

times = rnorm(100, 30, 1)
group = 1

data_3 = data.frame(times, group)



times = rnorm(100, 36, 1)
group = 0

data_4 = data.frame(times, group)

#create final dataset
final_data = rbind(data_1, data_2, data_3, data_4)

final_data$status = as.numeric("1")



#cohort level analysis (Kaplan-Meier)
km_trt_fit <- survfit(Surv(times, status) ~ group, data=final_data)


#plot results
autoplot(km_trt_fit)

enter image description here

In this hypothetical example, liver surgery is performed at time = 0.

  • We can see that after 15 years have passed since the surgery : the average probability of a woman surviving is roughly 50%, whereas the average probability of a man surviving is still at 100%.

  • However, after 30 years have passed since the surgery: The average probability of a woman surviving is still roughly at 50%, whereas the average probability of a man surviving is now at 25%.

Simply put, after liver surgery - women are at higher risk of "dying" compared to men, but as the years go on, men are at a higher risk of "dying" compared to women. Furthermore, we can see that confidence intervals generated for Men and Women at different times do not overlap, except for a brief period.

My Question: Based on these survival curves (assuming the data quality is suitable and the data is representative of the real world, negligible measurement errors, etc.), is it "reasonable" ("reasonable" solely based on the data and the results of the model, ignoring "ethical considerations" for the moment) to believe that :

  • In the first 20 years after liver surgery, "more" healthcare resources should be spent on women.

  • Between the 20th and the 30th year after livery surgery, "equal" healthcare resources should be spent on men and women.

  • After the 30th year, "more" healthcare resources should be spent on men.

I know that the above questions are very vague and too complicated to have straightforward answers - there is a whole field of "health economics" that studies these problems (e.g. How much is "more"? What about outliers? Cost of misdiagnoses? Benefit of intervention? What about QALY (Quality Adjusted Life Years)? Or perhaps more variables added to the model (e.g. Cox PH) might be able to give more realistic survival curves? Perhaps with more variables, more "precise" cohorts can be defined? Do some of these patients have additional underlying medical conditions that are both severe and undiagnosed that effect survival probability after liver surgery - thus skewing these survival curves? Are certain hereditary and lifestyle related factors influential on survival probabilities but not even recorded and taken into account within the models? etc.), using the above graphs, could such questions begin to be answered?

  • Suppose a hospital has a limited budget. And suppose after liver surgery, a patient either comes back to the hospital every year for a checkup or twice a year for a checkup (if considered at higher risk). Each checkup costs the hospital money, but let's assume that checkups have been proven to effectively bring attention to health complications (that arise from liver surgery) which left unaddressed can result in deaths. Could the above survival graphs be used to create and justify a policy that allocates budget for women to get more checkups in the immediate years after surgery, and fewer checkups for women in the later years after surgery?

Does anyone know if survival curves generated from the Kaplan-Meier method are used to make such decisions in the real world? (E.g. I imagine it must be more difficult when there are no clear patterns in the data and the confidence intervals often overlap between cohorts). Or are Survival Curves only used for "statistical inference" (e.g. What is the effect of smoking on survival after liver surgery for men? Is the effect of smoking "statistically significant"? ) ?

Can someone please comment on this?

Thanks!

References:

Note 1: I know that "hazard curves" are an integral part of Survival Analysis, and that hazard curves can be estimated by Kaplan-Meier - hazard curves provide valuable information on the "immediate hazard" of experiencing the event. However, explaining the concept of "hazard" to non-technical audiences is at times difficult compared to "survival probabilities" (e.g. survival curves monotonically decrease where as hazard curves can freely fluctuate). This is why I tried to frame my question only in the context of survival curves.

Note 2: I know that this is a hypothetical question I made up, but in the future, someone could also study the "impact of additional checkups on survival probabilities after liver surgery".

(NEW) Note 3: I am just thinking about an example - suppose I have one pot of water boiling on the stove, and another pot of water with sugar. Physics tells us that the pot containing water with sugar boils faster than the pot containing only water. So if I placed equal volumes of both liquids on the stove at the same time, it's more likely that the pot with sugar and water will boil and spill over compared to the pot with only water. One could make the argument that you should be "more attentive" to the pot with sugar/water compared to pot with only water, as the former is "riskier" than the latter. – The pot with sugar and water likely presents a greater "hazard" than the pot with only water. The pot with sugar and water is likely to "survive for a shorter time without overflowing" compared to the pot with only water. Could the argument be made that I should direct more of my attention to the pot with sugar and water until it boils, and after I have removed it from the stove - then direct more of my attention to the pot only containing water? This kitchen example seems like a fairly reasonable interpretation from a statistics standpoint. – Without taking into consideration the ethics of health and medicine - from a purely statistical standpoint : what is stopping us from interpreting and making decisions (e.g. prioritizing different cohorts of patients based on time-dependent risk) using the results of Survival Analysis?

$\endgroup$
4
  • 1
    $\begingroup$ Your question seems to confuse healthcare policy (management of healthcare systems and resources) with healthcare research (the development of new treatments). Decision making to determine policies and treatments have different goals and optimal decisions for a group (society) might not always align with optimal decisions for individual patients. $\endgroup$
    – dipetkov
    May 15 at 19:31
  • $\begingroup$ @ dipetkov: Thank you for your reply! I think I agree with you! have you ever come across some research/studies in which similar questions were asked as the question I posted? In general, what kind of approaches and models can be used to answer questions similar to the ones I asked? Thank you so much! $\endgroup$
    – stats_noob
    May 15 at 19:33
  • $\begingroup$ In truth I think your question is too simplified to discuss either policy or treatment in a very meaningful way. These are not questions but whole areas of study. And there are ethical aspects as well that are hard to quantify. So I agree with all the answers so far as they say that it takes lots more thinking than comparing two survival curves. $\endgroup$
    – dipetkov
    May 15 at 19:45
  • $\begingroup$ @dipetkov if you like, you could (gently) reframe the question as being whether the available data and summaries support the argument that OP's treatment strategy should be implemented. Like you're saying, the care actually given to a patient certainly cannot be determined solely through one data set. Note, my answer highlights that the data doesn't support the argument. $\endgroup$
    – Ben
    May 15 at 19:51

5 Answers 5

4
+50
$\begingroup$

The answer is unfortunately no. These curves tell you about the survival profile of men and women under the standard of care. On the other hand, your question is asking about an alternative of care. To understand whether a treatment strategy is effective, we have to understand how patients would respond to the care.

An answer to this question requires causal inference. Let $T_0$ denote the time of death for patients on the standard of care. Let $T_1$ denote the time of death for patients under the care defined by your 20 years/30 years after surgery policy. With this data, we can define an estimands such as \begin{align*} \psi_\mathrm{Male}(t) &= \mathbb{P}[T_1 > t \mid \mathrm{Male}] - \mathbb{P}[T_0 > t \mid \mathrm{Male}] \\ \psi_\mathrm{Female}(t) &= \mathbb{P}[T_1 > t \mid \mathrm{Female}] - \mathbb{P}[T_0 > t \mid \mathrm{Female}]. \end{align*} These estimands can be interpreted the expected improvement in survival among males and females after some given time $t$, if we were to implement your policy. If it happens that $\psi_\mathrm{Male}(t)>0$ for all $t$, so that survival is uniformly better among Males under your policy, and likewise $\psi_\mathrm{Female}(t)>0$ for all $t$, then there's a clear argument for using your policy. Males and females would be better off under your policy.

Your data can't be used to answer this question since there's no information about $T_1$, the time of death under your proposed policy. For example, maybe (for whatever reason) additional care would not help females survive in their first 20 years post-surgery.

$\endgroup$
9
  • $\begingroup$ @ Ben : Thank you so much for your answer! I think now I am finally beginning to understand the logic behind why Survival Curves can not be used for making these kinds of decisions : Because we do not have data on survival rates when these prioritization policies have been implemented. I guess the course of action would be to implement these prioritization policies on some sample of patients and then collect data to measure the effectiveness of these policies compared to existing policies? $\endgroup$
    – stats_noob
    May 15 at 18:52
  • $\begingroup$ Assuming I had such data about survival rates when these proposed policies were implemented on some samples of patients - what kind of models could be used to study their effectiveness when compared to the existing policy? In theory, could this still be done using Kaplan-Meier estimates? (i.e. compare the survival curve estimates of different cohorts of patients in the presence/absence of these policies?) Thank you so much! $\endgroup$
    – stats_noob
    May 15 at 18:55
  • $\begingroup$ This is the last question I hope to bug you with today - you seem to be really knowledgeable about Survival Analysis! This is a question that I have also tried to understand for a long time: Why do researchers tend to rarely use Survival Analysis to make estimates about individual patients ... and instead prefer using them to make estimates about cohorts of patients? I think this might be because of the "ecological fallacy" (i.e. the behavior of a group of patients tends to be more predictable than an a single patient)...but I am not sure if this is the reason. $\endgroup$
    – stats_noob
    May 15 at 19:00
  • $\begingroup$ here is the question: stats.stackexchange.com/questions/549382/… - can you please check this out at some point if you have time? Thank you so much! $\endgroup$
    – stats_noob
    May 15 at 19:00
  • $\begingroup$ (there is some research that has been done about this topic, e.g. jmlr.org/papers/volume21/18-772/18-772.pdf ... but I am not sure how "well-received" this research has been in the community) $\endgroup$
    – stats_noob
    May 15 at 19:07
2
$\begingroup$

I think that the answer is not easy and depends from a number of factors which cannot be captured only by the difference in survival curves.

First: the situation depicted is uttermost theoretical and I am having a hard time in hypotesize a real-world scenario resembling the curves that you produced. Indeed, there are situations in which the risk are "non-proportional" between groups, but almost always the differences are milder and not so significant.

Second: there may be several interacting terms that may influence the relationship depicted. For example, the difference in risk of mortality/outcome may not be directly related to sex, but to other confounders that are not taken into account in a simple Kaplan-Meier. You are saying:

(assuming the data quality is suitable and the data is representative of the real world, negligible measurement errors, etc.),

And this is clearly a very strong assumption here. Perhaps, one should investigate whether there is a difference in the current type of care which is offered to men and women (which may exert a role in creating the difference observed).

In short, it's very difficult to give a "yes" or "no" answer to your question.

On the other side, yes: the whole field of "Medical Economics" is devoted to investigate the cost-effectiveness of interventions (whether these are treatments or diagnostic procedures). And also yes, there are several tools to evaluate cost-efficacy of diagnostic procedures specifically. For example (which is obviously not exhaustive of the all options!) you can take a look to the Decision Curve Analysis. But there are many others.

$\endgroup$
4
  • $\begingroup$ Thank you for your answer! I added a new "Note 3" to my question - can you please take a look at it and let me know what you think? Thanks! $\endgroup$
    – stats_noob
    May 15 at 16:32
  • $\begingroup$ I was reading about Decision Curve Analysis - this looks very interesting! I think this might be more geared towards for deciding "thresholds for sensitivity testing" ... I am not sure if Decision Curve Analysis can be used for the problem that I am describing? Thanks! $\endgroup$
    – stats_noob
    May 15 at 16:33
  • $\begingroup$ @stats_noob Well, I think that your example it's more like of a 0/1 situation, while in medicine/health sciences this is almost never the case. However, there are situations in which we already "focus" our attention/effort on specific subgroups. Screening is one of that scenario: we do not screen all patients for colorectal cancer, but we do recommend to start at certain ages. There is plenty of works evaluating the cost-benefit of these approach and the "balance" in those. Incidentally, one of that takes into account sex-stratification (pubmed.ncbi.nlm.nih.gov/33533190). $\endgroup$ May 15 at 16:41
  • $\begingroup$ Thank you so much for your reply! I look forward to reading this paper you have linked! It sounds very relevant to the question that I have! $\endgroup$
    – stats_noob
    May 15 at 17:05
2
$\begingroup$

The short answer is "no"; healthcare resources are not allocated soley on the risk of mortality. In the analysis above, the cause for higher mortality was not identified. Thus, the hospital or medical team have more work to do to find and rectify the cause of higher mortality before resources are allocated. While useful, follow-up alone does not improve health unless it is tied to the delivery of important aspects of management (eg, diagnosis and treatment).

If I were presented with these curves, I would assume that some factor was present in the "0" group that was later present in the "1" group, and I would work to make certain that factor was identified and managed in all patients.

Update: There is no intrinsic flaw with the approaches you provide, but there are many reasons why this approach is not commonly or directly used as you describe. While any answer I provide will be incomplete, consider the following:

  1. Any risk factor identified should have a clear path to improving health. While not exactly analogous, this idea is similar to the notion of screening. See: https://www.ncbi.nlm.nih.gov/books/NBK138555/#!po=8.33333

  2. Despite any ideal test or risk factor, healthcare is incredibly complicated. For more insight, consider reading the 3-part New England Journal of Medicine series that begins with: https://www.nejm.org/doi/full/10.1056/NEJMms2200976

  3. Clinical reasoning takes into account many factors that include risk and many biases. For a short intro, see: https://www.aafp.org/afp/2011/1101/p1042.html

If none of these satisfy your curiosity, follow a doctor, nurse, or physician’s assistant for 2-3 days. My guess is that you will get a flavor for how medical literature influences the day-to-day practice and how it enters the 100s of decisions that occur (or don’t occur) every day.

$\endgroup$
7
  • $\begingroup$ @ Todd D : thank you for your answer! suppose the medical community believes that the designated cohorts ("red" and "blue" - women and men) are in fact the sole cause (i.e. "the factor") of higher mortality - could these curves then be used for allocation of resources? Or would hazard curves also be necessary? $\endgroup$
    – stats_noob
    Nov 22, 2021 at 16:40
  • $\begingroup$ I tried to create a generic question to illustrate allocation of resources using survival analysis. For example, instead of a hospital - imagine a used car dealership. The dealership wants to know which cars have longer life cycles and thus use survival analysis to estimate the life cycles for cohorts of cars (e.g. hybrids with 5,000 miles vs fully electric with 5,000 miles) ... the idea being, if a certain cohort (e.g. hybrid with 5000 miles) is identified to have a longer life cycle, they want to sell this cars from this cohort at a higher price. $\endgroup$
    – stats_noob
    Nov 22, 2021 at 16:43
  • $\begingroup$ Given a historic data set that shows how many days different used cars last before "failure" - can survival curves be used to help the dealership set the prices for different cars? $\endgroup$
    – stats_noob
    Nov 22, 2021 at 16:44
  • $\begingroup$ In general, causality or association are more important than the specific type of curve used to Deo TVT the relative difference between two groups. In you car example, I would say that advertising is a different goal than managing patients. I do not believe the analogy extends form cars to people. $\endgroup$
    – Todd D
    Nov 22, 2021 at 19:18
  • $\begingroup$ @ Todd: I added a new "Note 3" to my question - can you please take a look at it and let me know what you think? Thanks! $\endgroup$
    – stats_noob
    May 15 at 16:31
1
$\begingroup$

A somewhat sobering answer is that I believe these types of issues can often be ignored. For example, during grad school I did an internship at a large Pharma company. At one point, one of the senior statisticians referred to non-proportional hazards as a "purely academic issue", meaning they don't really worry about it at all.

Often times a target outcome is decided, such as median survival time, and then decisions are made based on statistical differences in this outcome. If non-proportional hazards are found, they may other reasons to post-hoc question the decision, but I'm not sure how often such concerns are directly built into the decision making process.

Similar to Todd's answer above, it's worth noting that very non-proportional hazards, such as your example, are often due to two groups facing differing underlying causes. So when designing a treatment, this is more of a corner case, although by no means impossible.

$\endgroup$
1
  • $\begingroup$ @ Cliff AB: Thank you for your answer! I added a new "Note 3" to my question - can you please take a look at it and let me know what you think? Thanks! $\endgroup$
    – stats_noob
    May 15 at 16:31
1
$\begingroup$

The type of inference you describe is one that would hinge on some implicit assumptions about the relationship between health risks and resources. Implicit in the suggestion to target funding to men/women at different times is the idea that the targetting of resources will tend to improve survival at around those times, and that ceteris paribus, greater resources will lead to a greater improvement. That is probably a reasonable "meta-medical" assumption in general, though it might be invalid in some cases. In any case, I think it is fair to say that this assumption is widely used in healthcare policy --- you see it regularly in political and managerial discussions of healthcare and it

As to whether this would actually be done in medical practice, I think that is possible. There are many cases in medical practice where medical research identifies different levels of risk amongst demographic groups for some particular affliction/condition, and resourcing is then targetted based on this knowledge. This is a complex field and involves interaction between ethical issues relating to demographic discrimination (e.g., race, sex, etc., discrimination), political considerations, and readings of medical studies. However, there are many cases where resources are indeed targetted to a particular demographic group due to underlying health research that shows that group to be at greater medical risk in some situation. Whether the underlying research specifically use a Kaplan-Meier estimator for a survival curve or something else is really only relevant insofar as this affects the credibility of the research; in principle there is no reason that statistical inference from a Kaplan-Meier estimator could not be used for this purpose.

I do not practice in the medical field and I have not seen any specific evidence of whether this particular case does or does not lead to differential targetting of resources to men/women. Nevertheless, the general policy of targetting resources to subgroups based on medical research is something that one can observe broadly in medical practice, so it is possible in principle.


Finally, since this is a statistics site, some notes on the analysis from a statistical perspective:

  • (1) Your plot should be improved by improving the labelling of variables, so that no textual clarification outside the plot is required. E.g., if you were to relabel "time" as "Years after liver surgery" then this would alleviate the need for accompanying textual explanation.

  • (2) Your plot should be improved by specifying the meaning of the shaded regions.

  • (3) The legend on your plot should refer to men and women, not a binary variable 0/1.

  • (4) It does not seem remotely credible to me that people have zero probability of death for years (or even one day) after liver surgery. This would make liver surgery the Holy Grail of medicine. Consequently, if you are making assumptions that lead to this conclusion, you should reconsider your entire statistical approach (and how silly it might look to anyone who is aware that humans are mortal).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.