1
$\begingroup$

I have a client that wants to test 3 pricing options to visitors of a website. All options are for the same product and service (no difference except pricing). Their current idea is to present all 3 options to visitors and use conversions to determine which is the right pricing model for their business.

I'm more familiar with split testing, and even then I don't have any experience designing tests outside of interfaces like Optimizely. I'd really like to understand this style of test better, and figure out how to determine the requisite sample size to achieve significant results.

Is there a name for this sort of test? What differences should I be aware of between an A/B test and this scenario?

Improve this question
$\endgroup$
5
  • $\begingroup$ Are you saying that a visitor will see one of 3 pricing options based on their business? Or are you saying each visitor will see all three options? In any case, what is your measure? Is it yes-no (purchase or not) or something else? $\endgroup$
    – Joel W.
    Oct 3, 2014 at 20:55
  • $\begingroup$ The method you describe has a bit in common with what's sometimes called monadic testing. Sources that go into that might also go into your method. $\endgroup$
    – rolando2
    Oct 3, 2014 at 22:38
  • $\begingroup$ @JoelW. Visitors will see all three options. The measure purchase or not, yes. $\endgroup$
    – coreyward
    Oct 3, 2014 at 22:42
  • $\begingroup$ I don't see how you can possibly learn what the profit-maximizing option is from this. Suppose I offer a large pizza for \$10, \$15, or \$20. Everyone picks the lowest price. That tells me nothing about what profits would be if I charged \$20 instead. At best, you get one point on the demand curve at p=$10 and learn nothing about the slope. $\endgroup$
    – dimitriy
    Oct 5, 2014 at 2:34
  • $\begingroup$ @DimitriyV.Masterov That's valid, but we're not looking to determine the optimal price for profit generation — we're testing demand for different pricing structures (e.g. rent vs buy). $\endgroup$
    – coreyward
    Oct 5, 2014 at 19:07

1 Answer 1

Reset to default
0
$\begingroup$

A chi-square goodness of fit test seems to be what you are looking for. It will tell you if the null hypothesis of equal preference for each option is tenable. As to sample size, look at this post on power: Power of chi-square test for goodness-of-fit as function of sample size

However, speaking practically, unless the preference for one option is overwhelming, omitting options that are not maximally chosen might still result in lost sales.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.