I have a dataset in which I have predicted risk probabilities from a survival model. The probability predictions were made at two different times for each observation - similar to a repeated measures outcome. I want to do a test of the mean to see if the predicted values change between timepoints.

Typically, I would used a paired t-test, but in this case the mean and the variance of the measurement are not independent due to being predicted risk probabilities from a survival analysis. The variance at the extreme ends of probability predictions is smaller than risk probabilities in the center - essentially a non-constant variance dependent upon where on the probability scale the measurement is located, which violates the t-test assumption that the mean and variance are independent.

What would be a good alternative to the paired t-test in this situation? One thought I had (after making this post originally) was to use a Wilcoxon Rank Sum on the differences. Being rank based, it is distribution free but certainly less efficient.

  • 1
    $\begingroup$ PLease give context and details of the real-world situation. Maybe a logistic regression? ... $\endgroup$ – kjetil b halvorsen Oct 26 '20 at 18:58
  • $\begingroup$ Survival modeling - predicting risk probabilities. $\endgroup$ – KirkD_CO Oct 26 '20 at 19:34
  • $\begingroup$ Actually, comparing changes in predicted risk probabilities over time. $\endgroup$ – KirkD_CO Oct 26 '20 at 19:53
  • $\begingroup$ What about Wilcoxon Rank Sums? Being rank-based, it is distribution free and should not have a requirement of independence of variance and mean as the t-test does. $\endgroup$ – KirkD_CO Oct 27 '20 at 2:34
  • $\begingroup$ Please edit your question to include more details (not just a few words, as with your comments) $\endgroup$ – Glen_b Oct 27 '20 at 10:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.