# Comparing Closeness Between Two Counts

I need to count the number of cells in several pictures. Since the picture is not totally clear, I let two people to count for each picture. I need to compare to see if the two counts are close enough so that I can use them, or else I would need a third count. What type of test can I use?

For example: In one picture, the counter counts the amount of cells in each sample to be 7 4 10 12 10. Another counter counts the same samples and give 9 8 12 13 10. How do I see if they are close enought?

PS I tried the paired T-test, it does not work well since it will return a high P value even if the two sets of data are 20 20 20 20 20 and 1 1 50 50 50.

• Nice question: You have good reason to doubt a paired t test will do what you want. A chi-squared test, based on counts, as in my Answer finds a clear difference between counts (18, 19, 20, 21, 22) and (1, 1, 50, 50, 50). – BruceET Oct 27 '19 at 20:40

I suggest a chi-squared test for homogeneity. Begin by making a table of observed counts as follows, with rows for your two counters, and five columns for the pictures. In R, you can make the table and do the test as follows:

a = c(7,4,10,12,10);  b = c(9,8,12,13,10)
TBL = rbind(a,b);  TBL
[,1] [,2] [,3] [,4] [,5]
a    7    4   10   12   10
b    9    8   12   13   10

output = chisq.test(TBL); output

Pearson's Chi-squared test

data:  TBL
X-squared = 0.96115, df = 4, p-value = 0.9156

The large P-value 0.9156, greater than 0.05, tells you that you cannot reject the null hypothesis that the two counters perform essentially the same.

You should look at a general explanation of this kind of chi-squared test is an elementary statistics text, or online. Here are some particulars of the R output for your data, which show how the test is done.

First, the observed counts can be echoed back for proofreading to make sure you put the correct counts into the computer.

output$obs [,1] [,2] [,3] [,4] [,5] a 7 4 10 12 10 b 9 8 12 13 10 Next, based on the null hypothesis, expected counts are computed. The expected count by the first counter for the first image is computed as follows. There are 43 counts in the first row and 95 counts altogether, thus the proportion of counts in the first row is $$43/95.$$ Also, the proportion of counts in the first column is $$16/95.$$ Thus, if the two counters are using the same method of counting, we would expect $$\frac{43}{95}\cdot\frac{16}{95}$$ of the total 95 counts to be in the upper-left cell of the table, and the expected count there should be $$95\cdot\frac{43}{95}\cdot\frac{16}{95}=7.242105.$$ Similarly, for the remaining nine cells of the table. output$exp
[,1]     [,2]      [,3]     [,4]      [,5]
a 7.242105 5.431579  9.957895 11.31579  9.052632
b 8.757895 6.568421 12.042105 13.68421 10.947368

Then the formula for the chi-squared statistic, 'X-squared` in the output above, is found by computing the 'contribution' $$\frac{(X_{ij} - E_{ij})^2}{E_{ij}}$$ for each cell. For cell $$(1,1).$$ that's $$\frac{(7-7.242105)^2}{7.242105} = 0.0081.$$ Similarly, for the remaining nine cells in the table. Thus the chi-squared statistic is the sum of ten such contributions to chi-squared:

$$Q = \sum_{ij}\frac{(X_{ij} - E_{ij})^2}{E_{ij}} = 0.96115.$$

Because all ten of the expected counts $$E_{ij} > 5,$$ the chi-squared statistic has approximately the chi-squared distribution with $$(r-1)(c-1) = (2-1)(5-1) = 4$$ degrees of freedom.

If the chi-squared statistic had been greater than the critical value $$c = 9.4877,$$ which cuts 5% of the probability from the right-hand tail of the chi-squared distribution, you would have rejected the null hypothesis (and sought another counter). The critical value from R is shown below, but you can find it in a printed table of chi-squared distributions. The critical value is shown as dotted red line in the figure below.

qchisq(.95, 4)
[1] 9.487729

The observed value of the chi-squared statistic (shown as a heavy vertical line in the plot below) is near the center of the distribution and not at all remarkable. This means that the corresponding observed and expected values in the table are in good agreement.

In the figure below, the P-value of the test is the area under the density curve to the right of the that line. You would have rejected the null hypothesis at the 5% level, if the P-value were smaller that 0.05. [Generally, printed tables of chi-squared distributions are not adequate for finding P-values; they are often given in computer output]

1- pchisq(0.96115, 4)
[1] 0.9156286

Notice that there are two (mathematically equivalent) indications to reject the null hypothesis at the 5% level: (a) The test statistic is greater than the 5% critical value, (b) The P-value is smaller than 5%.

T-test is used to compare the mean of two samples, which is not appropriate in your case. What you need is a method to compare the similarity of two sequence of number. You can compute the quantities like $$\sum_i |c_1^{(i)} - c_2^{(i)}|$$ or $$\sum_i |c_1^{(i)} - c_2^{(i)}| / |c_1^{(i)} + c_2^{(i)}|$$and set a threshold for recounting.