Conflicting answers in R depending on if the proportions test is two sided or one sided

Question

I want to investigate if the conversion rate(proportions) for system A is significantly different from the conversion rate of system B. $\mathcal{H}_0 = $ there is no significant difference of proportions and $\mathcal{H}_A = $ there is a significant difference between the proportions. If the $p-$value $< \alpha$ then $\mathcal{H}_A$ is true, if not then $\mathcal{H}_0$ remains true.

I have the following data:

            Sessions     Transactions     Proportions
System A:   90110        2296             0.02547997
System B:   93919        2186             0.02327538

For a two sided test at $\alpha = 0.05$ in in R I do:

sessions <- c(90110, 93919)
transactions <- c(2296, 2186)

res <- prop.test(transactions,sessions,correct = FALSE, alternative = "two.sided")
res$p.value

[1] 0.002162389

Since the $p-$value is less than $\alpha$ there is a significant difference between the two proportions

Question 1: In light of the above results, is it correct to conclude that: system A has a significantly greater conversion rate?

However, now I'd like to test if the conversion rate of system A is significantly lower than that of system B. That is I have $\mathcal{H}_0 = $ system A is significantly lower than system B and $\mathcal{H}_A = $ system B is significantly lower than system A. Conducting the same test, but with alternative = "less" I obtain

res <- prop.test(transactions,sessions,correct = FALSE, alternative = "less")
res$p.value

[1] 0.9989188

In this case, $p-$ value $> \alpha$ so $\mathcal{H}_0$ must remain true, so system A gives a lower conversion rate than system B.

I assume that the two sided test ONLY tells us that there exists a significant difference between the two, but not in which direction right? So the answer to Question 1 above should be no?

Answer 1

In the "two.sided" case, you are asking whether $0.02547997$ and $0.02327538$ are significantly different in either direction, and the test is saying "Yes, since the probability (if they are in fact equally likely) of seeing this result or one as or more extreme is about $0.00216$, which is unlikely. In this test "extreme" means a difference (the first minus the second) far away from $0$.

In your example of a one-sided case, you are asking whether $0.02547997$ is significantly "less" than $0.02327538$, and the test is saying "No, since the probability (if they are in fact equally likely) of seeing this result or one as or more extreme is about $0.99892$, which is likely. In this test "extreme" means a difference (the first minus the second) far below $0$.

If you had asked whether $0.02547997$ is significantly "greater" than $0.02327538$, and the test is saying "Yes, since the probability (if they are in fact equally likely) of seeing this result or one as or more extreme is about $0.00108$, which is unlikely and half the figure from the two-sided test. In this test "extreme" means a difference (the first minus the second) far above $0$.

The two-sided test rejects the null hypothesis of equality, and does so showing that the difference appears to be positive. The third test, if planned before seeing the data, would have done so more emphatically. So I would say that the answer to Question 1 is "Yes" on the basis of the first test and the data.