A chi-square or (stratified-)Cochran-Mantel-Haenszel test could be an option, but it would only test whether the proportion are the same. That's a bit of a limitation. Presumably you also care about a few other things:
- How much of a difference is there? Some differences might matter a lot, very small ones less so (one could even look at the cost trade-offs)?
- What drives any difference? This might help you figure out whether it's more to do with the process or more about other factors. Given COVID-19 and everything else that changes in the world, looking at just to the top-line numbers could be very misleading. For example:
- Consumers might have ordered more products they might have otherwise bought in a department store. That kind of customer might be more likely to return items, e.g. jewelry they could not see in person. Other items like, say, toilet paper are presumably a lot less likely to be returned.
- New customers might be trying out ordering things for the first time and might be more or less likely to return things.
From those perspectives, I'd try to model at a high level of granularity. A basic model for that might be logistic regression. It might even be important to consider whether certain individual customers (in case you know which sales are for the same customers) are more likely to return items and which items are more likely to be returned (not just broad categories). Then, something like a random effects logistic regression could be an option. E.g. to use R code something like
library(lme4)
glmer(cbind(no_returned, no_sold-no_returned) ~ (1|customer_id) + (1|product_id) + product_category + process,
binomial(link = "logit"))
would be a very simple model, but this is probably still too simplistic.
We still have to worry that the customers/situations under one process are different from those under another process (i.e. it's not a causal effect of the process, but rather about something else that is different with orders in one year vs. the other year).
So, if we truly wish to get at the causal effect of the process, we should consider causal inference techniques. E.g. one could use propensity scores for how representative orders are for one year vs. the other year (using all the possibly relevant information such as order history, transaction details etc. - in short anything you suspect might differ between the years - if you don't have all possible confounders, then that would be a limitation/a question mark on the conclusions), and adjust or stratify the analysis for these propensity scores. There's of course plenty of research ongoing on causal inference and you can make this as sophisticated as you like.
