I had a question regarding a new algorithm that processes MRI scans for the presence of infection and which sites of the body are infected.

I am assessing the negative predictive value of the algorithm in detecting infections (i.e. if the algorithm comes back as negative in 100 cases, how many of these are truly negative?) The output of the particular algorithm in question has to be interpreted by a radiologist, hence it's accuracy also depends on correct interpretation.

I've got a set of patients with proven infections and another group without proven infections, these will be age and sex matched. These cases have been retrospectively collected, hence the diagnosis has already been confirmed using gold-standard testing and the patients have already been treated (or not, depending on if they had infections).

I'll process their images using the new algorithm, collect output, anonymize and randomize said output and give them to 2 senior radiologists (who are familiar with the new algorithm) to correctly interpret. These radiologists will determine if the patient had an infection, and which sites of the body were infected. The interpreted results are then collected and processed to determine the negative predictive value of the new test.

I have some questions about this process: 1. What would you call this method? 2. Is it single-blinded or double-blinded? 3. How big should my sample size be to make this a statistically significant test? 4. How would I correlate between the two radiologists? 5. How do I calculate the negative predictive value?

Thank you for your help.

  • $\begingroup$ What is the fundamental output of the new test. Is it truly all-or-nothing? If not, then it is inappropriate to speak of 'negative' or 'positive'. $\endgroup$ – Frank Harrell Jan 28 '18 at 12:13
  • $\begingroup$ It's a diagnostic investigation - if it isn't negative or positive, I wouldn't have put up this question in the first place. $\endgroup$ – Code Monkey Jan 28 '18 at 18:45
  • $\begingroup$ My question is not about diagnostic tests or MRIs (I wouldn't put it up on this forum if it were). My question is about how to assess the negative predictive value of a test - it could be anything with a negative/positive result. It's a purely statistics-based question. Please assist. $\endgroup$ – Code Monkey Jan 28 '18 at 18:52
  • $\begingroup$ There's lot of material about that trivial case. NPV = P(disease absent | test is negative), computed directly if using a cohort study or using Bayes' rule if you have retrospective data. $\endgroup$ – Frank Harrell Jan 28 '18 at 20:47
  • $\begingroup$ How can it be called a retrospective cohort study? I am not studying risk factors, I'm assessing an investigation (i.e. I am not affecting patients, I am retrospectively looking through data to see if this algorithm can correctly identify patients with a disease). Basically, my first question is - what would you label this study as? Sorry for the questions - statistics is my Achilles heel... $\endgroup$ – Code Monkey Jan 28 '18 at 22:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.