I have a noob question for you. I have run two experiments: A and B. Between the two experiments I changed an element which should not change the results. I want to check that the results of experiment B do not contradict the results of experiment A.
The results for each experiment are a series of integer counts. I would expect that the results would be approximately normally distributed.
I was thinking of using the following method:
- Calculate mean and standard deviation of experiment A's results
- The null hypothesis is that the "independent" variable did not affect experiment B
- Use p-value and the normal distribution confidence interval [erf(n/sqrt(2))] to calculate the probability that a sample as extreme as the sample in B could appear in experiment A
- Multiply the calculated probability of all samples together to get the p-value
Is there a better approach?