In order to compare two lists of samples from 'before-treatment' and 'after-treatment', I am doing a two sample KS test using the ks_2samp function from Python's scipy.stats package which gives me the D and p-value statistics. Now I want to do this on a rolling basis. I periodically receive new samples and I want to find a way to reuse the KS statistics calculated from the previous set of samples.
E.g. if one set of before-treatment samples are in x and after-treatment in y then:
x = [1,1,1,1]
y = [2,2,2,1,1,1]
D, p_value = ks_2samp(x, y)
Output is:
D = 0.5, p_value = 0.43531018975534286
In next set of samples:
x = [1,1,1,1,1,1,1,1]
y = [2,2,2,2,2,1,1,1,1,1]
D, p_value = ks_2samp(x, y)
Output is:
D = 0.5, p_value = 0.14848228822491449
Note that for a given x and y, the number of 'before' samples in x and 'after' samples in y can differ. Also note that, the samples in x and y are from independent set of experiments.
Now I want to somehow combine the D and p_values from both of these separate ks test runs and get some idea of the overall result. Should I just take the mean or is there some other way that makes more sense? I don't have much knowledge of statistics so apologies if this is a very trivial question.
