I have assembled binary vectors (0/1 for all elements and equal weight and arranged in time order) that have been separated into different cohorts where a unique event of interest occurs. I have removed the event of interest element itself and the prior 3 months of elements from all vectors. Now, I take a new vector to test and calculate the average pairwise Jaccard similarity between this vector and each cohort individually.

My questions center on interpretation:

What is the statistical interpretation of an average pairwise Jaccard similarity score in this example? Can this be seen as a probability or not?

If the number of samples in these cohorts increase, can it be interpreted that this would improve the prediction?

If this is valid, what would be the best performance metrics for evaluating this (Precision/Recall, F Score, Cross Validation?

Any advice would be sincerely appreciated. I'm just curious if this idea might be useful as an alternative to traditional survival analysis/time-to-event in my use case.

  • $\begingroup$ What does "statistical interpretation" mean? Can you be specific about what kind of thing you seek? I've been a statistician for quite a long while but have no clue what you mean by that. $\endgroup$ – Glen_b Aug 21 '16 at 4:07
  • $\begingroup$ @Glen_b Thank you kindly for your clarification inquiry. I guess the root of my inquiry is really centered on whether a jaccard similarity coefficient in the use case I detailed would be considered a truly valid probability measure. $\endgroup$ – Pylander Aug 21 '16 at 7:27
  • $\begingroup$ @Glen_b Say for example, that my example test vector when compared to a very large cohort gave an average pairwise jaccard similarity of 0.85. Should I interpret this to mean that this test vector is simply 85 percent similar to the average in the cohort? Or could I potentially go further with a sufficiently large sample in the cohort and say that there is a probability of 85% that the event of interest will occur. This is the crux of it I believe. $\endgroup$ – Pylander Aug 21 '16 at 7:28
  • $\begingroup$ Perhaps -- if you get a set with similarity of 0.85, it does have 85% of the union in the intersection -- you might say that's "85% overlap" and you could reasonably define that as "percentage similarity". You could then argue that the average similarity was 85%. $\endgroup$ – Glen_b Aug 21 '16 at 8:55
  • $\begingroup$ @Glen_b That definitely sounds reasonable as the basic interpretation. I guess I had just been hoping the since these cohorts are defined by the occurrence of an "event of interest". That I might be able to say "85% similar OR probability of event x occurring"? $\endgroup$ – Pylander Aug 21 '16 at 17:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.