0
$\begingroup$

I am trying to compare multiple algorithms using the Friedman test and post hoc Dunn test. I know that we should use the rank of our data before performing the Friedman test, but should we rank the data for Dunn test too? I am using python to perform the tests. Here is my code:

import pandas as pd
import scikit_posthocs as sp
print("Q synthetic statistical test:")

q_synthetic = {
"DATA_1": [0.61,0.54,0.59,0.52,0.59,0.54,0.62,0.52,0.49,0.50,0.47,0.62,0.60,0.56],
"DATA_2": [0.43,0.43,0.24,0.24,0.43,0.43,0.43,0.24,0.43,0.43,0.43,0.43,0.43,0.43],
"DATA_3": [0.82,0.82,0.64,0.61,0.82,0.74,0.82,0.74,0.82,0.82,0.82,0.81,0.82,0.82],
"DATA_4": [0.70,0.58,0.10,0.20,0.70,0.58,0.70,0.70,0.70,0.70,0.70,0.69,0.70,0.70],
"DATA_5": [0.53,0.11,0.10,0.10,0.53,0.10,0.53,0.53,0.53,0.53,0.53,0.53,0.53,0.53],
"DATA_6": [0.66,0.11,0.10,0.10,0.66,0.18,0.66,0.66,0.66,0.66,0.66,0.66,0.66,0.66]
}

dataframe_q_synthetic = pd.DataFrame.from_dict(q_synthetic, orient='index',
                                   columns=["ALGO_1", "ALGO_2", "ALGO_3", "ALGO_4", "ALGO_5", "ALGO_6", "ALGO_7","ALGO_8", "ALGO_9", "ALGO_10", "ALGO_11", "ALGO_12", "ALGO_13", "ALGO_14"])
general_dataframe_q_synthetic = dataframe_q_synthetic
l_data = pd.melt(general_dataframe_q_synthetic, var_name='criteria', value_name='score')
p_values_general_q_synthetic = sp.posthoc_dunn(l_data, val_col='score', group_col='criteria', p_adjust='bonferroni')    
res = friedmanchisquare(general_dataframe_q_synthetic["ALGO_1"],general_dataframe_q_synthetic["ALGO_2"],general_dataframe_q_synthetic["ALGO_3"], general_dataframe_q_synthetic["ALGO_4"], general_dataframe_q_synthetic["ALGO_5"] ,general_dataframe_q_synthetic["ALGO_6"], general_dataframe_q_synthetic["ALGO_7"], general_dataframe_q_synthetic["ALGO_8"], general_dataframe_q_synthetic["ALGO_9"], general_dataframe_q_synthetic["ALGO_10"], general_dataframe_q_synthetic["ALGO_11"], general_dataframe_q_synthetic["ALGO_12"], general_dataframe_q_synthetic["ALGO_13"], general_dataframe_q_synthetic["ALGO_14"])
print("friedman test: ")
print(res)
print("algorithms p_values")
print(p_values_general_q_synthetic)

It returns all 1.0 matrix which means there is no significant difference between any of the groups while the Friedman test shows the existence of a significant difference. Therefore, I suppose there should be some mistake. Now, should I change my input to ranks? For example, should I write my data like this:

q_synthetic = {
    "DATA_1": [12, 6, 9, 4, 9, 6, 13, 4, 2, 3, 1, 13, 11, 8], ...

Which one is correct?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.