Return to Question

clarified question in title; added tag; light editing

Source Link

edited Apr 12, 2017 at 16:35

147.5k
89
406
717

Is Chi Squarethe chi-squared test correct hereto see if different algorithms differ in output?

I want to test if the outcomes of different algorithms are different with statistical significance. I am testing 4 algorithms, each of which output 0 or 1 after every run.

I am running these algorithms for multiple series of multiple runs, e.g. 2 series of 5 runs in this example. So, the output of each algorithms looks like this (using Python):

alg1 = array([[0, 0, 1, 1, 1], [1, 1, 0, 1, 0]])
alg2 = array([[1, 1, 0, 0, 0], [1, 1, 1, 1, 1]]) 
alg3 = array([[0, 0, 1, 1, 1], [1, 0, 1, 0, 0]])
alg4 = array([[1, 0, 0, 1, 1], [1, 1, 0, 1, 1]])

My idea was to use the Chi Squaredchi-squared test to see if the outcomes are different. Again, using Python:

obs = np.array([alg1, alg2, alg3, alg4])
scipy.stats.chi2_contingency(obs)

Is the Chi Squaredchi-squared test the correct test to see if the outcomes of these algorithms are independent?

Is Chi Square test correct here?

I want to test if the outcomes of different algorithms are different with statistical significance. I am testing 4 algorithms, each of which output 0 or 1 after every run.

I am running these algorithms for multiple series of multiple runs, e.g. 2 series of 5 runs in this example. So, the output of each algorithms looks like this (using Python):

alg1 = array([[0, 0, 1, 1, 1], [1, 1, 0, 1, 0]])
alg2 = array([[1, 1, 0, 0, 0], [1, 1, 1, 1, 1]]) 
alg3 = array([[0, 0, 1, 1, 1], [1, 0, 1, 0, 0]])
alg4 = array([[1, 0, 0, 1, 1], [1, 1, 0, 1, 1]])

My idea was to use the Chi Squared test to see if the outcomes are different. Again, using Python:

obs = np.array([alg1, alg2, alg3, alg4])
scipy.stats.chi2_contingency(obs)

Is the Chi Squared test the correct test to see if the outcomes of these algorithms are independent?

Is the chi-squared test correct to see if different algorithms differ in output?

I want to test if the outcomes of different algorithms are different with statistical significance. I am testing 4 algorithms, each of which output 0 or 1 after every run.

I am running these algorithms for multiple series of multiple runs, e.g. 2 series of 5 runs in this example. So, the output of each algorithms looks like this (using Python):

alg1 = array([[0, 0, 1, 1, 1], [1, 1, 0, 1, 0]])
alg2 = array([[1, 1, 0, 0, 0], [1, 1, 1, 1, 1]]) 
alg3 = array([[0, 0, 1, 1, 1], [1, 0, 1, 0, 0]])
alg4 = array([[1, 0, 0, 1, 1], [1, 1, 0, 1, 1]])

My idea was to use the chi-squared test to see if the outcomes are different. Again, using Python:

obs = np.array([alg1, alg2, alg3, alg4])
scipy.stats.chi2_contingency(obs)

Is the chi-squared test the correct test to see if the outcomes of these algorithms are independent?

Source Link

asked Apr 12, 2017 at 15:31

JNevens

Is Chi Square test correct here?

I want to test if the outcomes of different algorithms are different with statistical significance. I am testing 4 algorithms, each of which output 0 or 1 after every run.

I am running these algorithms for multiple series of multiple runs, e.g. 2 series of 5 runs in this example. So, the output of each algorithms looks like this (using Python):

alg1 = array([[0, 0, 1, 1, 1], [1, 1, 0, 1, 0]])
alg2 = array([[1, 1, 0, 0, 0], [1, 1, 1, 1, 1]]) 
alg3 = array([[0, 0, 1, 1, 1], [1, 0, 1, 0, 0]])
alg4 = array([[1, 0, 0, 1, 1], [1, 1, 0, 1, 1]])

My idea was to use the Chi Squared test to see if the outcomes are different. Again, using Python:

obs = np.array([alg1, alg2, alg3, alg4])
scipy.stats.chi2_contingency(obs)

Is the Chi Squared test the correct test to see if the outcomes of these algorithms are independent?

chi-squared-test python independence