One way ANOVA on compliance rates (by time period) with different lengths

Question

I'm stumped a little on this, as in my stats classes we had very nicely packaged data with nice, normal data that was easy to understand! Apologies if this seems totally obvious.

My data is a series of compliance rates among several categories of staff over several audit periods (AP). We'll say the action they are supposed to perform is swiping a card when they arrive at work. Compliance rate is, of course, # of times they actually swiped the card / # of times they were expected to.

I have three date groups to compare, but all of them vary in length, as we're trying to show during and post-COVID compliance. Data is measured every four months, so there are three APs in a year, starting in November:

• AP 1: 1 November to 31 March
• AP 2: 1 April to 30 June
• AP 3: 1 July to 31 October

I'm observing the data from November 2018 to October 2022, split as such:

Date Group 1 (pre-COVID, baseline/control):

• AP-1, November 2018 - 31 March 2019
• AP-2, 1 April 2019 - 30 June 2019
• AP-3, 1 July 2019 - 31 October 2019
• AP-1, 1 November 2019 - 31 March 2020 [12 months, 4 periods total]

Date Group 2 (COVID):

• AP-2, 1 April 2020 - 30 June 2020
• AP-3, 1 July 2020 - 31 October 2020
• AP-1, 1 November 2020 - 31 March 2021
• AP-2, 1 April 2021 - 30 June 2021
• AP-3, 1 July 2021 - 31 October 2021 [20 months, 5 periods total]

Date Group 3 (post-covid):

• AP-1, 1 November 2021 - 31 March 2022
• AP-2, 1 April 2022 - 31 June 2022
• AP-3, 1 July 2022 - 31 October 2022 [9 months, 3 periods total]

The data looks something like this for all categories of staff combined.

Group	AP code	Card swiped	Total entries	Compliance
1	2019-1	1019	1231	0.790
1	2019-2	782	878	0.788
1	2019-3	934	1132	0.793
1	2020-1	973	1151	0.821
2	2020-2	640	749	0.834
2	2020-3	901	1075	0.810
2	2021-1	952	1122	0.816
2	2021-2	674	807	0.800
2	2021-3	841	1001	0.804
3	2022-1	727	857	0.820
3	2022-2	733	887	0.784
3	2022-3	868	1041	0.801

I want to use ANOVA to see if there was a significant different in compliance rate from date group 1 (pre-COVID) to date groups 2 and 3 (COVID and post-COVID). Would using one-way ANOVA be the solution? Asking because I'm not accustomed to dealing with rates, rather "here's how long the leaf petals are." Working in R if that helps. Thanks heaps!

Answer 1

You could do an ANOVA of

lm(compliance ~ group)

and then create a contrast of 1 vs. 2 and 3.

This answers your question as posed. But you might consider using period (or even month), as opposed to group and then using a spline of period. This would let you see whether things changed at specific points, and how those relate to COVID.

Not only does this allow a finer look at what was going on, but it also does not categorize COVID as "pre" "in" and "post". I think that categorization is a bad idea. Reactions to COVID, and rules about it, and so on all changed continuously. E.g. people didn't wake up on Nov. 1 and say "oh good! COVID is over! Let's all go back to normal."

Answer 2

As per the previous answer, yes, you can use an ANOVA (DV=compliance, continuous variable; IV=Group, catehorical, with 3 levels).
But... Your sample size is very small (4, 5 and 3). It is unlikely you have enough power to find any significant difference.
Even for some form of regression analysis, the sample size is limited.
I assume the compliance value is an aggregate value, over many employees? Can you access the compliance data per employee? Or per employee group? This would give you a (much?) larger sample size in each group, and hence a better chance of seeing a difference (if it is there).