Statistical significance between several percentages

Question

I am trying to find a statistical difference between the success rate of several machine learning techniques using different activation functions. I obtained the success rates for each machine learning technique and the different activation functions. The following data is what I obtained:

+-------------+-----------------------+-----------------------+-----------------------+
|    Column   | Activation function A | Activation function B | Activation function C |
+-------------+-----------------------+-----------------------+-----------------------+
| Algorithm 1 | 90%                   | 70%                   | 50%                   |
| Algorithm 2 | 40%                   | 50%                   | 100%                  |
| Algorithm 3 | 60%                   | 90%                   | 90%                   |
+-------------+-----------------------+-----------------------+-----------------------+

As you can see there is one entry with a success rate of 100%, however that does not necessarily mean that algorithm 2 with activation function C is definitely the best approach. It might be the case that 1A, 2C, 3B and 3C are all good options since there might not be a significant difference among these algorithms. Which significance test should I use to test which algorithm(s) in combination with an activation function(s) is preferred over others?

The sample size is fairly small, per condition n=10.

Answer 1

As a simple start, convert your percentages to counts, then you get an contingency table and can use a chisquare test. I show below how to do this in R:

> x <- matrix(scan(), 9, 2, byrow=TRUE)
1: 9 1 4 6 6 4 7 3 5 5 9 1 5 5 10 0 9 1
19: 
Read 18 items
> x
      [,1] [,2]
 [1,]    9    1
 [2,]    4    6
 [3,]    6    4
 [4,]    7    3
 [5,]    5    5
 [6,]    9    1
 [7,]    5    5
 [8,]   10    0
 [9,]    9    1
> chisq.test(x)

    Pearson's Chi-squared test

data:  x
X-squared = 18.93, df = 8, p-value = 0.01524

Warning message:
In chisq.test(x) : Chi-squared approximation may be incorrect
> chisq.test(x, sim=TRUE, B=10000)

    Pearson's Chi-squared test with simulated p-value (based on 10000
    replicates)

data:  x
X-squared = 18.93, df = NA, p-value = 0.0148

So, yes, you can conclude that there are really some differences in success probabilities. To investigate that further you could try logistic regression.