How do I run a correlation analysis on medical diagnosis (Dx) codes between two years at patient level. Data runs into 45000+ observations for each patient with different diseases across years and I am trying to establish a correlation using some non parametric approach.

My data would look like:

patient  2013    2014

1       25000   25080
1       49121   49122
1       44490   44490

likewise for another 1000 patients. The codes given are Dx based on the ICD 9 CMS guidelines.

  • 1
    $\begingroup$ Are you interested in whether the Dx for 2013 is independent of the Dx for 2014, or if the two years agree? Do you want a measure of association / agreement, or do you need a test, or both? $\endgroup$ – gung - Reinstate Monica Feb 23 '15 at 5:16
  • 1
    $\begingroup$ Dx in 2014 is dependent on Dx in 2013 because the Dx codes are chronic in nature (meaning once they are found in a person, it will persist forever), based on the medical records i have, i need to see whether the doctor diagnosed the patient with the Dx that he had last year as well. i am not looking at any tests for now, i looked at Kappa but for some reasons the results i get are not stronger though i have Dx codes present in both years. hope this clarifies $\endgroup$ – learning_stats Feb 23 '15 at 5:24

It sounds like your situation is closer to agreement than correlation. Something like Cohen's kappa is probably right. I gather you are disappointed because your results were non-significant. My guess is that you have very many diagnosis categories, perhaps almost as many as patients. This can decrease your statistical power. If you don't need a test, you could just calculate the proportion of Dx's that match in both years.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.