Skip to main content
added 488 characters in body
Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k

Your suggestion should work.

I'm going to make another suggestion, which also yields an integer value for the discrepancy from uniformity. As indicated in comments, we don't really have enough information to say whether it's better for your application.

The usual chi-squared goodness of fit statistic is $\sum_i (O_i-E_i)^2/E_i$ (where $O_i$ is the observed count in category $i$ and $E_i$ is the expected count). When used for deviation from perfect uniformity, $E_i=N/k$, where $N=\sum_i O_i$ is the total count and $k$ is the number of categories.

This chi-squared statistic from uniformity is also related to the simple variance of the counts.

Note that this statistic simplifies in the uniformity case, as follows:

\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}

A simple linear rescaling of the chi-squared statistic is then $\sum_i O_i^2$, which will be integer-valued.

With $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ (the average count rounded down) into $k-r$ bins and $\lceil N/k \rceil$ (the same, rounded up) into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts. This would give an arrangement like $[1,2,1,2,2]$ get the value $0$, since it cannot be made smaller. If you'd like such an arrangement to get a non-zero value, the value of $\sum O_i^2$ under exactly equal allocation is $N^2/k$, but this won't be an integer in such cases, so you'd need to round that down before subtracting from $\sum O_i^2$ (rounding down would mean the difference $(\sum O_i^2)-\lfloor N^2/k\rfloor$ would only be exactly zero when the spread was perfectly uniform).

The usual chi-squared goodness of fit statistic is $\sum_i (O_i-E_i)^2/E_i$ (where $O_i$ is the observed count in category $i$ and $E_i$ is the expected count). When used for deviation from perfect uniformity, $E_i=N/k$, where $N=\sum_i O_i$ is the total count and $k$ is the number of categories.

Note that this simplifies in the uniformity case, as follows:

\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}

A simple linear rescaling of the chi-squared statistic is then $\sum_i O_i^2$, which will be integer-valued.

With $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ (the average count rounded down) into $k-r$ bins and $\lceil N/k \rceil$ (the same, rounded up) into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts. This would give an arrangement like $[1,2,1,2,2]$ get the value $0$, since it cannot be made smaller. If you'd like such an arrangement to get a non-zero value, the value of $\sum O_i^2$ under exactly equal allocation is $N^2/k$, but this won't be an integer in such cases, so you'd need to round that down before subtracting from $\sum O_i^2$ (rounding down would mean the difference $(\sum O_i^2)-\lfloor N^2/k\rfloor$ would only be exactly zero when the spread was perfectly uniform).

Your suggestion should work.

I'm going to make another suggestion, which also yields an integer value for the discrepancy from uniformity. As indicated in comments, we don't really have enough information to say whether it's better for your application.

The usual chi-squared goodness of fit statistic is $\sum_i (O_i-E_i)^2/E_i$ (where $O_i$ is the observed count in category $i$ and $E_i$ is the expected count). When used for deviation from perfect uniformity, $E_i=N/k$, where $N=\sum_i O_i$ is the total count and $k$ is the number of categories.

This chi-squared statistic from uniformity is also related to the simple variance of the counts.

Note that this statistic simplifies in the uniformity case, as follows:

\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}

A simple linear rescaling of the chi-squared statistic is then $\sum_i O_i^2$, which will be integer-valued.

With $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ (the average count rounded down) into $k-r$ bins and $\lceil N/k \rceil$ (the same, rounded up) into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts. This would give an arrangement like $[1,2,1,2,2]$ get the value $0$, since it cannot be made smaller. If you'd like such an arrangement to get a non-zero value, the value of $\sum O_i^2$ under exactly equal allocation is $N^2/k$, but this won't be an integer in such cases, so you'd need to round that down before subtracting from $\sum O_i^2$ (rounding down would mean the difference $(\sum O_i^2)-\lfloor N^2/k\rfloor$ would only be exactly zero when the spread was perfectly uniform).

added 488 characters in body
Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k

The usual chi-squared goodness of fit statistic is $\sum_i (O_i-E_i)^2/E_i$ (where $O_i$ is the observed count in category $i$ and $E_i$ is the expected count). When used for deviation from perfect uniformity, $E_i=N/k$, where $N=\sum_i O_i$ is the total count and $k$ is the number of categories.

Note that this simplifies in the uniformity case, as follows:

\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}

A simple linear rescaling of the chi-squared statistic is then $\sum_i O_i^2$, which will be integer-valued.

With $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ (the average count rounded down) into $k-r$ bins and $\lceil N/k \rceil$ (the same, rounded up) into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts. This would give an arrangement like $[1,2,1,2,2]$ get the value $0$, since it cannot be made smaller. If you'd like such an arrangement to get a non-zero value, the value of $\sum O_i^2$ under exactly equal allocation is $N^2/k$, but this won't be an integer in such cases, so you'd need to round that down before subtracting from $\sum O_i^2$ (rounding down would mean the difference $(\sum O_i^2)-\lfloor N^2/k\rfloor$ would only be exactly zero when the spread was perfectly uniform).

The usual chi-squared goodness of fit statistic is $\sum_i (O_i-E_i)^2/E_i$ (where $O_i$ is the observed count in category $i$ and $E_i$ is the expected count). When used for deviation from perfect uniformity, $E_i=N/k$, where $N=\sum_i O_i$ is the total count and $k$ is the number of categories.

Note that this simplifies in the uniformity case, as follows:

\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}

A simple linear rescaling of the chi-squared statistic is then $\sum_i O_i^2$, which will be integer-valued.

With $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ (the average count rounded down) into $k-r$ bins and $\lceil N/k \rceil$ (the same, rounded up) into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts.

The usual chi-squared goodness of fit statistic is $\sum_i (O_i-E_i)^2/E_i$ (where $O_i$ is the observed count in category $i$ and $E_i$ is the expected count). When used for deviation from perfect uniformity, $E_i=N/k$, where $N=\sum_i O_i$ is the total count and $k$ is the number of categories.

Note that this simplifies in the uniformity case, as follows:

\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}

A simple linear rescaling of the chi-squared statistic is then $\sum_i O_i^2$, which will be integer-valued.

With $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ (the average count rounded down) into $k-r$ bins and $\lceil N/k \rceil$ (the same, rounded up) into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts. This would give an arrangement like $[1,2,1,2,2]$ get the value $0$, since it cannot be made smaller. If you'd like such an arrangement to get a non-zero value, the value of $\sum O_i^2$ under exactly equal allocation is $N^2/k$, but this won't be an integer in such cases, so you'd need to round that down before subtracting from $\sum O_i^2$ (rounding down would mean the difference $(\sum O_i^2)-\lfloor N^2/k\rfloor$ would only be exactly zero when the spread was perfectly uniform).

added 685 characters in body
Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k

To expand my updated comment into an answer, a simple linear rescaling of aThe usual chi-squared goodness of fit statistic is $\sum_i (O_i-E_i)^2/E_i$ (where $O_i$ is the observed count in category $i$ and $E_i$ is the expected count). When used for deviation from perfect uniformity, $E_i=N/k$, where $N=\sum_i O_i$ is the total count and $k$ is the number of categories.

Note that this simplifies in the uniformity case, as follows:

\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}

A simple linear rescaling of the chi-squared statistic is then $\sum_i O_i^2$, which will be integer-valued.

If $N=\sum_i O_i$, and $k$ is the number of groups, andWith $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ (the average count rounded down) into $k-r$ bins and $\lceil N/k \rceil$ (the same, rounded up) into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts.

To expand my updated comment into an answer, a simple linear rescaling of a chi-squared statistic for deviation from perfect uniformity is $\sum_i O_i^2$, which will be integer-valued.

If $N=\sum_i O_i$, and $k$ is the number of groups, and $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ into $k-r$ bins and $\lceil N/k \rceil$ into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts.

The usual chi-squared goodness of fit statistic is $\sum_i (O_i-E_i)^2/E_i$ (where $O_i$ is the observed count in category $i$ and $E_i$ is the expected count). When used for deviation from perfect uniformity, $E_i=N/k$, where $N=\sum_i O_i$ is the total count and $k$ is the number of categories.

Note that this simplifies in the uniformity case, as follows:

\begin{eqnarray} \sum_i (O_i-E_i)^2/E_i &=& \sum_i (O_i-N/k)^2/(N/k)\\ &=& \frac{k}{N} \sum_i (O_i-N/k)^2\\ &=& \frac{k}{N} \sum_i [O_i^2-2N/k\cdot O_i+(N/k)^2]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k \sum_i O_i+\sum_i (N/k)^2)]\\ &=& \frac{k}{N} [\sum_i O_i^2-2N/k\cdot N+ k\cdot(N/k)^2)]\\ &=& (\frac{k}{N} \sum_i O_i^2)-2N+ N\\ &=& (\frac{k}{N} \sum_i O_i^2)-N \end{eqnarray}

A simple linear rescaling of the chi-squared statistic is then $\sum_i O_i^2$, which will be integer-valued.

With $r={N\mod k}$, you could compute the smallest possible such value by putting $\lfloor N/k\rfloor$ (the average count rounded down) into $k-r$ bins and $\lceil N/k \rceil$ (the same, rounded up) into $r$ bins. It would be reasonable - but not necessary - to subtract the sum of squared counts for this arrangement from the above sum of squared counts.

deleted 3 characters in body
Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k
Loading
Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k
Loading