2
$\begingroup$

I am trying to calculate survival rate for all grades (Kindergarten to 1st Grade) of school. See the example below for Grade 1 students. Y0 refers to number of students enrolled in grade 1 in that particular year. Y1 : how many of them are still in the school 1 year later. These students would have moved to grade 2 in Y1. Similarly for the other years. I have similar data for other grades as well for years 2010 to 2019. As of now I am calculating survival rate based on (number of students in Yn divided by number of students in Y0). For example : In Y2 survival rate is column sum of Y2 divided by column sum of Y0 (excluding 2018 as data is not available in Y2 for 2018). It is 76%. I will do the same calculations for other Yn periods and for other grades and then average out all the annual rates.

I wanted to know is there any better way to calculate survival rate more statistically?

$\endgroup$
2
  • $\begingroup$ Dividing one column sum by another is biased. After all, Y0 includes students in kindergarten in 2018 while Y2 has yet to reflect the advancement of any of them to second grade. You have, in effect, supposed that all 114 kindergartners in 2018 have been lost, which is unlikely unless the school shut down before 2020. $\endgroup$
    – whuber
    Jul 13, 2020 at 21:45
  • $\begingroup$ I am excluding 114 in denominator. Any alternative approach you suggest? $\endgroup$
    – john
    Jul 13, 2020 at 21:58

1 Answer 1

4
$\begingroup$

The Kaplan–Meier estimator for survival probability, $S[t]$, is computed as a product of survival probabilities over disjoint sub-intervals $$ S[t] = \prod_{k \le t} s_k $$ where $s_k$ is the probability to survive over $t=[k-1,k)$.

In your case, the atomic sub-intervals correspond to columns of your table $n_{:,k}$.

For a single row (i.e. cohort), $i$, you could estimate the interval probabilities as $$ s_k^{(i)} = \frac{n_{i,k}}{n_{i,k-1}} $$ and in this case, the Kaplan-Meier estimate $$ S_k^{(i)} = \frac{n_{i,1}}{n_{i,0}}\times\frac{n_{i,2}}{n_{i,1}}\times\cdots\times\frac{n_{i,k-1}}{n_{i,k-2}}\times\frac{n_{i,k}}{n_{i,k-1}} = \frac{n_{i,k}}{n_{i,0}} $$ reduces to a simple ratio, due to cancellation of successive factors.

However to use all of the data, here you want to pool or aggregate the different cohorts (rows). As noted in the comments, you cannot simply sum each column, because for later cohorts the data is censored.

For computing aggregate interval probabilities, both numerator and denominator should be summed over the same "lumped cohort" (set of rows), corresponding to where there is data for both columns.

Note that once you do this, the product of the interval probabilities will no longer reduce to a simple ratio (i.e. the denominator for one interval will no longer cancel with the following numerator).

The need for the multiplicative approach is illustrated below (using your table): Comparison of KM vs. naive estimates. The blue curve uses the Kaplan–Meier approach, while the orange curve uses a single ratio of current year to Y0 (both summed over rows with current-year data). Note that while the two are similar, the naive ratio approach shows a logically impossible increase in survival from Y8 to Y9. In contrast, the Kaplan–Meier estimate correctly shows constant survival from Y6 to Y9 (i.e. since after Y6, there is no decrease in survivors in any row).


Example Calculations

(Note: survival at Y0 is $s_0 = 1$, by definition.)

Survival from Y0 to Y1 ... $$ s_1 = \frac{\sum_{2010\le{i}\le2018}n_{i,1}}{\sum_{2010\le{i}\le2018}n_{i,0}} = \frac{3+\cdots+114}{3+\cdots+98} = \frac{599}{686} $$

Survival from Y1 to Y2 ... $$ s_2 = \frac{\sum_{2010\le{i}\le2017}n_{i,2}}{\sum_{2010\le{i}\le2017}n_{i,1}} = \frac{3+\cdots+104}{3+\cdots+114} = \frac{435}{501} $$

...

Survival from Y5 to Y6 ... $$ s_6 = \frac{\sum_{2010\le{i}\le2013}n_{i,6}}{\sum_{2010\le{i}\le2013}n_{i,5}} = \frac{2+\cdots+28}{2+\cdots+31} = \frac{59}{68} $$

...

Survival from Y8 to Y9 ... $$ s_9 = \frac{\sum_{2010\le{i}\le2010}n_{i,9}}{\sum_{2010\le{i}\le2010}n_{i,8}} = \frac{2}{2} $$

(Note: $s_7 = s_8 = s_9 = 1$, since there is no rightward decrease in students from Y6 onward.)

Similarly, $S_0 = s_0$, $S_1 = s_0s_1$, $S_2 = s_0s_1s_2$, etc. For example $$ S_2 = 1 \times \frac{599}{686} \times \frac{435}{501} \approx 0.758 $$

$\endgroup$
9
  • $\begingroup$ Thank you. Would you mind calculating the same for the above table for demonstration purpose? Or for 1-2 years? Thanks in anticipation! $\endgroup$
    – john
    Jul 17, 2020 at 20:23
  • $\begingroup$ Was my answer understandable? As one conceptual demonstration: in your data, all students who make it to Y6 survive. Is that clear? $\endgroup$
    – GeoMatt22
    Jul 17, 2020 at 21:38
  • $\begingroup$ Thank you. I mean if you show calculations for a few years that would be a great help. How to pool different cohorts and calculate aggregate survival rate $\endgroup$
    – john
    Jul 18, 2020 at 6:11
  • 1
    $\begingroup$ @john sorry for changing to subscript notation without warning! Yes, the capital $S_k$ numbers are survival from $0$ to $k$. (The lowercase $s_k$ numbers are survival from $k-1$ to $k$.) BTW if my answer has addressed your original question, you can accept it. $\endgroup$
    – GeoMatt22
    Jul 19, 2020 at 23:25
  • 1
    $\begingroup$ This assumes the survival probability from age $k-1$ to $k$ is a single consistent number. Here I assumed for example that the underlying survival rate did not vary by "cohort". There are approaches to check/test if survival varies by e.g. cohort/grade/school. But I am not the best to advise. I would suggest you post a new question framing the over-arching research question you want to address, and summarizing your data. Then you may get better answers! $\endgroup$
    – GeoMatt22
    Jul 19, 2020 at 23:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.