I've started to study feature selection techniques and i have a situation that I don't understand. I've created a synthetic dataset with 5 predictor variables and a label, the predictor variables are as follow: X1 = random boolean (p=0.5) X2 = random boolean (p=0.5) X3 = random boolean (p=0.5) X4 = XOR(X2, X3) X5 = random boolean (p=0.5) Y = XOR(X1, X2, X3) So when I apply mutual information criteria to rank the relevance of the variables I'm getting that X5 is the most relevant, as far as I know it should be the lower in the rank because Y does not depend at all of it. Here is my code: ```` import numpy as np from sklearn.feature_selection import mutual_info_classif as MIC from Py_FS.filter import Relief SAMPLE_SIZE = 100 def xor_three(a, b, c): return (~a*~b*c)+(~a*b*~c)+(a*~b*~c)+(a*b*c) def xor_two(a, b): return a != b X1 = np.random.choice([False, True], size = SAMPLE_SIZE) X2 = np.random.choice([False, True], size = SAMPLE_SIZE) X3 = np.random.choice([False, True], size = SAMPLE_SIZE) X4 = xor_two(X2, X3) X5 = np.random.choice([False, True], size = SAMPLE_SIZE) X = np.column_stack([X1,X2,X3,X4,X5]) Y = xor_three(X1, X2, X3) mi_score = MIC(X,Y, discrete_features=True) print(mi_score) ```` As an example in one of the execution I've got in mi_score -> [0.00163452 0.00068499 0.00501039 0.01996895 0.02920959] Not sure if I've missundertood the concept or I am doing something bad.