I am hoping someone can either validate something I think is wrong or explain to me why it is not.
Here is the situation
- On a marketing test a given selection criteria for customers was tested, and a random control group was kept (of all customers who qualified, a random was sample was the control). The test group was called and the control was not.
- a given response was measures (purchase or not) and statistics aggregated
- an hypothesis test was performed to see if calling increase the purchase rate (2 proportion test); however
- The person who did this multiplied the size of the control group and the number of responders for the control by a factor - the factor was roughly the one required so that the test group was the same size as the control group. Then they performed the test and reported the result
My position is that this is incorrect, you can't adjust the size of the control after the test. They could have kept a larger control of course. I think the problem was the Ns were too small and hence the result were not significant and they really wanted to show them to be significant
Is there a logical statistical explanation that makes this valid? I do not believe so as if multiplying by say 5 does not work you can multiply by 10 or 100. My best guess is they were trying to do something like oversampling the control group
Thanks in advance