# Does mutual information depend on the number of data points?

I am playing with mutual information in scikit-learn.

import numpy as np
from sklearn.feature_selection import mutual_info_regression

X = np.linspace(-10,10,101)
y = X**2
print('MI:', mutual_info_regression(X.reshape(-1, 1),y)[0])


This gives me the result

MI: 1.759


Then I change the number of points,

X = np.linspace(-10,10,1001)


I get

MI: 3.918


For 10001 points:

MI: 6.189


For 100001 points:

MI: 8.483


Why is that? Should it really depend on the number of points or I am doing something wrong? Mutual information is a measure of the mutual dependence between the two variables. I would expect from such a measure some fluctuation when the number of points is small. But here, it looks like it monotonously increases with the number of points.

• I'm not familiar with that particular software implementation, but I'm guessing that what is happening is that the entropy is changing as you add more data because it is categorizing one or both of the inputs rather than treating them as truly discrete. Your variables are extremely dependent on one another so your MI estimate is going to be near the entropy. Dec 27, 2018 at 20:41