# Doubts with P-value and Hypothesis in Permutation Test problem

I have to solve this problem for a stats course:

An e-commerce company is testing a new design for a web page. The objective is to achieve at least a 2% increase in the conversion rate.

An experiment has been designed with two groups:

• Control Group A

• Treatment group B

The table with data has 30 rows and looks like this:

In order to solve this we are requested to define the hypotheses and test them with a permutation test, which I did.

I defined the hypotheses as follows:

• $$H_0: \mu_{B} - \mu_{A} \geq 2\%$$
• $$H_1: \mu_{B} - \mu_{A} < 2\%$$

And this were my results:

My P-value is almost 0, which rejects the null hypothesis.

I'm not sure if I'm formulating my hypotheses correctly as I actually want to confirm my $$H_0$$ instead of rejecting it and I have doubts if my P-value is being calculated correctly.

Help pls!

I would get the mean delta for B-A, subtract off 0.02, and called this $$\mu_{obs}$$. Then during $$B=1000$$ iterations, get the mean delta of B-A with column A shuffled (permuted), and call this $$\mu^{(b)}$$. The p-value will be equal to the number of times $$\mu^{(b)}$$ is greater than $$\mu_{obs}$$, divided by $$B$$. You know that when you shuffle column A and then take the difference between B and A, that average will probably be zero. You also know that B-A for the observed data (unshuffled), $$\mu_{obs}$$, may be 0.02 or more. Thus you have to subtract off the 0.02, and this way you will be testing that the two "zero means" means are significantly different.