# Hypothesis Test Question

I am hoping someone can either validate something I think is wrong or explain to me why it is not.

Here is the situation

1. On a marketing test a given selection criteria for customers was tested, and a random control group was kept (of all customers who qualified, a random was sample was the control). The test group was called and the control was not.
2. a given response was measures (purchase or not) and statistics aggregated
3. an hypothesis test was performed to see if calling increase the purchase rate (2 proportion test); however
4. The person who did this multiplied the size of the control group and the number of responders for the control by a factor - the factor was roughly the one required so that the test group was the same size as the control group. Then they performed the test and reported the result

My position is that this is incorrect, you can't adjust the size of the control after the test. They could have kept a larger control of course. I think the problem was the Ns were too small and hence the result were not significant and they really wanted to show them to be significant

Is there a logical statistical explanation that makes this valid? I do not believe so as if multiplying by say 5 does not work you can multiply by 10 or 100. My best guess is they were trying to do something like oversampling the control group

• When you say "multiplied the size" do you mean they simply used a different $n$ in the calculations to the $n$ that was actually sampled? Or do you mean they went back and sampled some more to increase $n$? (as ghonke appears to think). Both are wrong, but one is - in some sense at least - a substantially greater level of fakery and flim-flam than the other. Feb 4, 2015 at 0:03
• Yes say the control group was N=100 and had 5 responders, they multiplied by say 5 and use N=500 and 25 responders and they they did the test with whatever the test had Feb 4, 2015 at 0:41
• If they genuinely think it's a good idea to do that, a simulation might be the best way to convince them otherwise. Feb 4, 2015 at 19:27
• agree, i actually built something quick for using coin tosses to make sure I was no missing anything Feb 5, 2015 at 1:22