Why does chi2-test show me a dependence between randomly generated columns? I generate two columns of length 343180 with random integer values between 0 and 290 and run sklearn's chi2-test of dependence. One would expect that the null hypothesis (independence) is accepted with a high probability, but actually I get a test score of approx. 15423 and a p-value of 0.
import numpy as np
from sklearn.feature_selection import chi2

X = np.transpose([[np.random.randint(0, 291) for i in range(0, 343180)]])
y = np.asarray([np.random.randint(0, 291) for i in range(0, 343180)])

print(X.shape)
# output: (34318, 1)

print(y.shape)
# output: (34318,)

chi2(X, y)
# output: (array([15423.73497325]), array([0.]))
# which means: p-value = 0.

Does this has to do with the limits of pseudo random number generation? Or do I misunderstand the concept of a chi2-test? Does the chi2-test, as implemented in sklearn, expect a certain type of distribution of the tested features, and not just an arbitrary discrete distribution?
 A: I think you are right to doubt the output. I tried to replicate the result by implementing the Chi-square test as it is supposed to work for categorical data, e.g. see this link. I don't get the result that comes from sklearn.feature_selection.chi2, here's my code:and outputs:
import numpy as np
from scipy.stats import chi2
from sklearn import feature_selection 
np.random.seed(1)

c = 291
n = 343180
X = np.random.randint(c,size=(n,1))
y = np.random.randint(c,size=(n,1))

print('chi2 test stat, pval:',feature_selection.chi2(X, y))

# assuming classes are in rows, calculate expected for rows
p_class = np.zeros((c,1), dtype=np.double)
for yi in y:
  p_class[yi] = p_class[yi] + 1
p_class = p_class / n

n_feat = np.zeros((c,1))
for x in X:
  n_feat[x] = n_feat[x] + 1

exp = p_class @ n_feat.T

# feature levels are columns  
obs = np.zeros((c,c))
for i in np.arange(n):
  obs[y[i],X[i]] = obs[y[i],X[i]] + 1


#print(exp)
#print(obs)

stat = np.sum((obs-exp)**2/exp)
dof = (c-1)**2
chi2crit = chi2.ppf(1-0.05,df=dof)
pval = 1-chi2.cdf(stat,df=dof)
print('my stat,pval,crit(5%):',stat,pval,chi2crit)


chi2 test stat, pval: (array([13790.24430418]), array([0.]))
my stat,pval,crit(5%): 84025.71482666291 0.5712498907190868 84775.72566291611

A: Three issues:

*

*Software engineering: Avoid 'magic numbers' in the code. Define the constants in the code and re-use them. That makes it easier to change the code to run with different values.


*Self-education: Have you tried changing any of your 'magic numbers'?


*Statistics: $\chi^2$-test checks for differences in ratios (relative frequencies). It would be quite a surprise if the ratios of so many random numbers would be the same.
So, to give you an answer by addressing all three points:
import numpy as np
from sklearn.feature_selection import chi2

U = 291
L = 0
N = 2 # used to be 343180
X = np.transpose([[np.random.randint(L, U) for i in range(0, N)]])
y = np.asarray([np.random.randint(L, U) for i in range(0, N)])

print(X.shape)
# output: (2, 1)

print(y.shape)
# output: (2,)

chi2(X, y)
# output: (array([7.42424242]), array([0.00643509]))
# which means: p-value < 0.007.
```

