# Detect dramatic increase in proportion among multiple classes

I would like to use some tool to detect spike in one class's proportion.

Assume I received roughly the same percentage of red, blue and yellow candies throughout time. That means the absolute number can go up and down for each type of candy but the percentage for each candy will stay roughly the same. For example, it could be

1. 80 red (28%), 90 blue (32%), 110 yellow (39%)
2. 1 red (33%), 1 blue (33%), 1 yellow (33%)

And these are totally fine.

I want a tool to statistically detect when a particular percentage is larger than usual. For example, let's say one day i received 5 red (71%), 1 yellow and 1 blue. That will trigger the warning that the percentage for red is abnormal.

My gut instinct told me i should use Chi-square test to test one class vs. the rest of the classes. Is that correct?

• It would be hard to answer this without knowing a little more about your problem but maybe you just need a control chart? rss.onlinelibrary.wiley.com/doi/10.1111/j.1740-9713.2004.024.x Commented Mar 26, 2022 at 18:04
• Control charts for proportions: itl.nist.gov/div898/handbook/pmc/section3/pmc332.htm Commented Apr 8, 2022 at 11:19
• @kjetilbhalvorsen - that nist handbook is golden. @ Devin, I'm a different breed of nerd, so standard disclaimers apply. I would look at confidence interval for a sample proportion when thinking about it, you need a sample size large enough to say the CI for the sample is less aligned with what would be expected when drawn from the same population. Personally, I would use bootstrap simulation to get a bottom-up sense of what the values should be, or should not be, assuming you had sufficient population sizes. Commented Apr 8, 2022 at 11:41

First transform your observations into proportions, as you have mentioned. Then you can apply some outlier detection method. Using z-score or Median Absolute Deviation on each class will probably work quite well for a few classes.

Note: for N classes, the proportions can be fully described by N-1 numbers. Since the proportions sum to 100%. So one column is a redundant column and can be dropped.

• Transforming to proportions looses to much information, 1/10 and 10/100 are the same proportions, but the uncertainty is vey different! Commented Apr 8, 2022 at 11:17
• Fair point! One could add the total as a separate feature and also detect anomalies in that. This would be an ML type approach, not so statistically strong. Commented Apr 8, 2022 at 11:48

Your goal is: "I want a tool to statistically detect when a particular percentage is larger than usual." That is what outlier/anomaly detection methods are made for.

Those methods have different techniques for achieving this. There are good libraries available, see e.g. here for python libs. Among the most popular methods, that also work for a large number of classes, are e.g. k-nearest neighbor methods or isolation forest. Note, however, that they rather detect any kind of isolation, which might also be an isolated very small value as for instance the value $$0.1\%$$ in $$(0.1\%, 24.9\%, 25\%, 25\%, 25\%)$$. Or it could be an isolated value in the middle. Those anomalies would not give you high values of z-score.

Most anomaly detection methods provide you with an outlier score which is a measure of its "outlier-ness". That makes those methods particularly flexible since you can yourself decide about where to set the threshold, i.e. above which value of the outlier score a value is to be considered an outlier.

So here (below) is a customizable tool (in R) that you can explore your problem with, and it is how I would think about the problem using a bottom-up approach. You can re-run it a few times with the same settings to get a feel for what run-to-run differences look like and what your uncertainty in the estimate of the means might look like.

This link is to a paper on abrupt change detection and is how I like to think about detecting abrupt changes.

Code:

library(tidyverse)
library(matrixStats)

#parameters
num_boots <- 3000
num_draws <- 10

#make original population
pop <- sample(x = c("a", "b", "c"), #put your values here
size=1000,            #total population
prob=c(1,1,1)/3,      #sum to 1, frequencies of values
replace = T)

#convert to factor (makes compute convenient)
pop <- as.factor(pop)

#get stats on it
summary(pop)

#prep for loop
log_store <- matrix(NA, num_boots, length(levels(pop))) %>% as.data.frame()

#bootstrapping
for(i in 1:num_boots){
y <- sample(pop,num_draws)

for(j in 1:length(levels(pop))){
log_store[i,j] <- length(which(y==levels(pop)[j] ) )

}
}

#make summaries in a useful form
df <- data.frame(labels = levels(pop),
sample_mean    = colMeans(log_store),
sample_std_dev = colSds(log_store %>% as.matrix()),
sample_max = colMaxs(log_store %>% as.matrix()),
sample_min = colMins(log_store %>% as.matrix()) )

#print it
print(tibble(df))


My output looks like this:

So what do I get from this?

• mean number per draw should have about the same proportions as in population
• one standard deviation is about 1.5 units so, if I use rules of thumb, about 67% of the time my values should be within 1.5 units of the mean, and about 99% of the my values should be within about 4.5 units of the mean.

To get the 80/90/110 you would change the code to

pop <- sample(x = c("red", "blue", "yellow"), #put your values here
size=280,            #total population
prob=c(80,90,110)/280,      #sum to 1, frequencies of values
replace = T)


If you only draw 7 per day then adjust 'num_draws' to be 7 instead of ten.

The result from that would be this:

From this we see that most of the time we get 2-3 of each. We would find it rather unlikely to get more than about 6 red candies.