Significance Between Two Percentage Decreases Over Time, Not Mutually Exclusive Variables and Different Sample Sizes

Question

I work in an healthcare improvement department.

I have spent a large amount of time looking for the best way to test this problem for significance and have come up short.

If you take a look at this screenshot of run charts (control charts):

I am basically looking for the best way to determine if the two percentage changes in the medians (from start point to end point of each chart) are significantly different (the decrease being 37.5% in one chart vs 33.3% in the other).

The way the data is collected is that on one day each month every patient in the hospital is assessed for urinary tract infection (UTI) and also if they have a urinary catheter. Each person surveyed may have none, just a UTI, just a catheter or both. This measurement is then converted into a proportion of n for each month and plotted as a data point.

Each month (each data point), n will be different depending on how many people are in the hospital at that time. n is also different at each data point between the two charts (Jul 12 in the left chart will have different n to the point at Jul 12 on the right) This occurs for 48 months.

I have tried looking at different tests - chi squared, McNemar's, t-tests and nothing seems to fit. I think my problem is that I don't know what to use for n, and then what test to use to compare the percentage decreases in median over the 48 months using this n.

Answer 1

What you need, is an actual control chart or Shewhart chart. In this case, you will want to use a $p$ chart.

The control chart plots $p$, the ratio of defects (patients with a UTI, for example) over the total (patients in the ward/hospital). A centerline is created, $\overline{p}$ that is the mean of all $p$ values. Control limits are calculated for each data point at $\overline{p}\pm 3s$ and are based upon the sample size (20% of 5 patients having a UTI is very different from 20% of 2,000 patients having a UTI). The equation for $3s$ is $$3\sqrt{\frac{(\overline{p} * ( 1 - \overline{p})}{n}}$$ Points outside of the control limits would be determined to be statistically significant differences (along with other standard tests which can be used to determine if a process is out of statistical control.