4
$\begingroup$

I want to conduct an A/B test between two different Ad Copies I have for a given advertisement on Google.

Ad Copy A might have 30 k impressions (number of times ad was displayed) with approximately $700 in revenue.

Ad Copy B might have 60 k impressions with approximately $600 worth of revenue.

Originally I thought I should use a chi-squared distribution -- but thought against it as the variables used, were not discrete but continuous (revenue could be from 0 to infinity).

I was looking online and I saw something regarding a MWW test (Mann-Whitney U Test) used for testing web pages vs. the revenue per visitor.

Is this the correct test to use for my split? Does anyone have any other suggestions regarding what statistical test I can use to reject my null hypothesis(H0: there is no statistically significant difference between the two ad copies; or in other words, A and B cannot be compared because such differences in performance could be due to experimental/sampling size errors).

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

Yes, you'll want to use the Mann-Whitney U test, because revenue is a continuous variable, and because the distribution of revenue is not normal, it's typically skewed right, with a lot of $0 values. (If it were normally distributed, you would use a t test of independence.)

A chi-squared goodness of fit test makes sense in an A/B test for a proportion, such as number of conversions per impression.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Because of the central limit theorem, the quantity OP is interested is approx normally distributed for the large number of samples used. see eg semanticscholar.org/paper/… $\endgroup$
    – seanv507
    Commented Feb 20, 2019 at 14:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.