0
$\begingroup$

In a usual online marketing set up - a typical application of AB test is:

impression / clicks = CTR%

I am trying to investigate a different set up:

revenue_amount / conversion = conversion_value (monetary)

Can I run a test on the metric 'conversion_value'.

The revenue is generate taking into account the number of conversions - imagine - in a subscription model set up.

I ask myself - does it make sense to run a test on this kind of value? The revenues and the amount of conversions are in intrinsically correlated.

$\endgroup$
2
  • $\begingroup$ Isn't the click-through rate (CTR) defined as clicks divided by impressions? You have it as the reciprocal. More to the point, what do you mean by "run a test"? A significance test? If so, what null hypothesis would you want to test? Or if you meant an experiment ("A/B test"), which variable did you want to manipulate and which variable did you want to examine as an outcome? $\endgroup$ Nov 29, 2016 at 19:22
  • $\begingroup$ yes edited that, CTR is clicks/impression. But forget about that. I want to test the hypotesis that banner A is better than B in term of conversion_value. The test runs like that. A, B banners are the same: at the time t one of the B receives a change. After x days the test runs and they must be evaluated. So my question is: does it sense to use the metric revenue/conversions? $\endgroup$ Nov 30, 2016 at 8:32

1 Answer 1

0
$\begingroup$

If the revenue_amount and conversion are perfectly correlated, then you will get the same result for both of your test cases A & B. This is not a bad metric by itself to know from a business perspective, but this will not help you differentiate between your test cases.

$\endgroup$
1
  • $\begingroup$ The correlation matrix of conversions and conv_value for banner A,B is: Corr_ab = (1,-1,-1,1). $\endgroup$ Nov 30, 2016 at 8:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.