Statistical test on dataset containing percentage data that do not add up to 100% I am trying to test for significance on a set of counts. In a normal scenario, I could use a chi-square test to test each condition, however, I have an issue that the percentages do not add up to 100. My real problem actually comes from baseball data that I am analyzing from a motor control perspective. I am trying to make the argument that certain variables cannot account for significant changes in homerun percentage, specifically, different pitch parameters (type of pitch, velocity, movement, etc.).
For clarity, I will simplify the question with a more general toy problem. A common example for chi-square is where you have two classes and are counting the number of instances of each grade.





A
B
C




Class 1
20
32
15


Class 2
24
28
16




In the case above, I can use chi-squared to compare class A to class B. However, what if I had a bunch of classes and I only wanted to compare based on the # of A's. Furthermore, each year the class is a different size.





A
B
C
# Students




Class 1
20
32
15
67


Class 2
24
28
16
68


Class 3
18
32
12
62


Class 4
21
35
15
71


...
...
...
...
...


Class N
25
30
14
69




It doesn't have to be this way, but it could also be represented as % data since the classes are different sizes each year:





A
B
C




Class 1
0.30
0.48
0.22


Class 2
0.35
0.41
0.24


Class 3
0.29
0.51
0.19


Class 4
0.30
0.49
0.21


...
...
...
...


Class N
0.36
0.43
0.20




NOTE: These are all made up numbers and are just to explain the problem.
So my question then is how can I compare these A's since they all come from different class sizes? Chi-square seems wrong and I assume that I cannot simply use a t-test for this problem since the data are not normally distributed and are a bit more uniform with some slight variations (also there is a small number of groups, or classes as in above). One solution that has been proposed to me by a friend is to use Bayes factors to determine if certain values (classes in the above example) deviate from the overall average. I.e., making the case for each class being represented by individual %'s vs the classes having values near the overall group mean.
A second component to this question is more specifically related to the baseball problem. Let's say that I want to consider the relationship between velocity and homerun %. As opposed to pitch types, which is naturally discrete (analogous to class number above), velocity can be treated as continuous. The velocity can be binned and analyzed in the same manner or perhaps a different approach exists.
BTW, all of my analysis is currently in python.
Thanks!
 A: I’m struggling with a similar situation here so I’ll share my reasoning in hope we can discuss it further. :)
The general idea is to go for a permutation test over the percentage distribution (I’m not sure if we can frame the problem this way, though).
The idea is to decide which statistic to test upon, e.g, using the mean difference of the percentage distribution - in your case A and not A.
Note that this is different from saying that the mean percentage is such a value, instead, it is the mean of the percentage distribution.
I’m not sure if is there any problem in following this approach. I hope someone can help and provide more details on this.
Once this is possible, one example of permutation test Python (taken from the Practical Statistics for Data Scientists book) is:
import random

def perm_func(x, nA, nB):
    """A Python version of a permutation test.
    
    This function works by sampling (without replacement) nB indices and assigning
    them to the B group; the remaining nA indices are assigned to group A. The
    difference between the two means is returned.

    Parameters
    ----------
    x: dataframe
        A single data set represented by a dataframe object containing all the samples
    nA: int
        number of samples to resample from group A
    nB: int
        number of samples to resample from group B
    Return
    ------
    float
        The difference between the two means
    """  
    n = nA + nB
    idx_B = set(random.sample(range(n), nB))
    idx_A = set(range(n)) - idx_B

    return x.loc[idx_B].mean() - x.loc[idx_A].mean()


