I have a dataset and I want to compute the mutual information (MI) for a selected set of variables. The dataset is large enough so that computation of the MI may take undesirably long time. Can I just take a 10% sample of the data and use it to compute the MI measure?
I am aware of different sampling techniques including random sampling, stratified sampling and bootstrapping. Even though techniques that require computing MI more than once such as bootstrapping or clustering provide good clues about bias and variance, they lead to some extra computational time. I want to compute MI only once. So, preparing a single random sample and using it for estimating MI is a solution. My concern is how accurate is this solution based on a sample size of 10% of data? What about the sampling error and do I have to pay attention to it?
Note: Regarding computing the MI, I do not assume normality for estimating any probability; rather, a non-parametric method (i.e. histogram) is used.