# How to calculate whether a disease outcome in a population is by chance or significant?

I have a huge dataset of longitudinal medical records. Each record contains indications for drug and disease.

I have been able to calculate the Outcome Rate in the population given a particular drug. This is, simply 1 minus the ratio number of diseases after the number of diseases before the drug.

Unfortunately, this calculation would suggest that every drug I am interested in has a poor level of medical efficacy. It is possible that the drugs I am interested in might only work on a subset of patients, therefore, a complete population-based study is going to be very biased.

Given how many patients show a reduction in disease after taking the drug, how can I determine whether that was statistically significant or just by chance? I would like to learn a little more about how I can compare groups of those affected by the drug and those not affected.

• Are the drugs you are interested in even intended to cure anything? Insulin is great for type 1 diabetes, but it will never cure it; the same is true for most drugs for chronic conditions. Their efficacy is about preventing bad outcomes, not a cure. Antibiotics, on the other hand, are practically miracle-level drugs for curing many infections. – Bryan Krause Oct 5 '18 at 22:05
• Sorry, you are quite right. I've changed the text to "outcome" rather than "cure". The disease of interest will typically always show signs of either chronic relapse or long-term recurrence e.g., migraine. – Anthony Nash Oct 5 '18 at 22:19
• I was attempting to take a before after ratio from a givne drug date. Interestingly the average rate in disease indications (a note on a patient's file) goes up. i.e., the population gets worse. Then it occured to me that also need to drill down on those individuals who despite their cohort-neighbours, are actually getting better. – Anthony Nash Oct 5 '18 at 22:23
• Part of the problem you are probably running in to is that those who are taking drugs to treat something are probably doing worse than those that aren't, which is why people do randomized trials rather than cohort studies to look at drug effects. – Bryan Krause Oct 5 '18 at 22:58
• Thanks, Bryan. I think that probably explains what I am seeing. – Anthony Nash Oct 5 '18 at 23:02