# Making sure I'm normalizing my data appropritely

I'm (actually) evaluating a sorting machine's accuracy. For simplicity, let's say I have 10 bins. Each day, a random number and assortment of objects are inducted into the machine and sorted. Sometimes the machine is inaccurate, so to test its accuracy, I sample random number of objects from each bin. While doing so, I record the number of objects I evaluated, and the number of those that were incorrectly sorted. In addition, at the end of the sort, I take the real distribution that day of how many objects of each type should have been sorted into each bin.

The first thing I'm currently doing is taking the total number of objects evaluated in the sample, and finding the number of incorrect objects as a percentage of that total. This is the raw, unweighted accuracy calculation. In the real world, it is useful for troubleshooting, so I want to keep that data.

But what I also want to do is have a weighting system. For instance, if 50% of the objects were sorted into bin 1, and 2% were sorted into bin 4, but there were more mis-sorted objects in bin 4, it stands to reason that those mis-sorted objects should be weighted less overall than the bin that had many more objects correctly sorted to it. Extrapolating this logic over millions of objects sorted, it makes sense that one bin should have a bigger impact on accuracy than the other.

What I want to do is make sure I'm weighting these bins correctly. Currently this is what I'm doing:

(Bin, Sampled objects, Mis-sorted Objects, Real Distribution): (1, 4, 0, 14%); (2, 6, 0, 10%); (3, 3, 1, 14%); (4, 2, 0, 11%); (5, 8, 0, 13%); (6, 7, 0, 7%); (7, 1, 2, 8%); (8, 15, 1, 12%); (9, 12, 0, 4%); (10, 6, 0, 7%);

Given the sample data, if I want to normalize these bins, I'm currently doing the following: Multiply the sampled objects by the real distributions, multiply the mis-sorted objects by the real distributions, sum those new respective values and retake the percentages.

Is this sensible? Or am I doing something statistically dumb?

• Can you explain the "real distribution" percentage? Bin 1 has 4 sorted objects for 14%, and bin 2 has 6 sorted objects for 10%? What is that percentage? How is it computed? Commented Aug 2 at 3:26
• The induction system is tallying this real distribution. Maybe that's not a good name. Imagine object 137 is inducted. An algorithm tells the sorting machine where that object should go. Whether it gets there correctly or not depends on the machine performance. So at the end of the sort, we know the percentage breakdown of where every object should have gone. That's what I'm defining as the real distribution. Commented Aug 2 at 11:54