1
$\begingroup$

I'm currently facing the following situation. We have run a marketing campaign providing to some members one of two type of coupons. In addition to this, some of these members were already contacted in the previous one by another campaign.

So I have the following dataset, where:

-mixpre3M= is a flag variable telling us if the customer had already purchased something in the past

  • bono recibido: is the kind of coupon recived by the customer. The type "3euros" is the coupon identifying the customers already touched in the past by a campaign. the type "benchmark" identify the customer who haven't received any coupon (control group)

  • tran_during: is the N of redeemers or purchasers

  • enviados: is the number of people included in each group

mixpre3M bono_recibido TRAN_DURING_CAMP_FLG enviados

  0     benchmark                 5948                 33336
  1     benchmark                  557                 2102
  0    BONO3EUROS                   96                 1233
  1    BONO3EUROS                   17                 83
  0    BONO6EUROS                 4823                 25434
  1    BONO6EUROS                  626                 1793

What I want achive is if there is a redemption or purchasing rate significatively different between each group, and see between which group there is difference

Now, I have the following doubts:

a. I understand I should run a multiple comparison test, like maybe a GLM with binomial distribution, but I'm not sure it is a correct procedure, considering that some of the groups (for instance the fourth one, with n=83) are quite smaller than the main other groups. Is the model I choose the correct one for this kind of analysis, and I should exclude the smaller groups?

b. I understand this is a kinda similar to a multiple A/B test. Does anyone know any tutorial or material which could help with the topic? Never managed this kind of test before

Improve this question
$\endgroup$
1
  • $\begingroup$ Yes, probably the best approach is to use multiple logistic regression (as you say, a glm with a binomial distribution). If you are using modern software, there should be a post-hoc prodedure available to make any comparisons you would like. Depending on your software, you may need to change the data so that you have a column for Redeemed? with a value of 1 or 0, on separate rows, and the counts for each , e.g. 5948, 27388. This Redeemed? variable would be your dependent variable. I don't think the smaller groups will be too problematic. $\endgroup$
    – Sal Mangiafico
    Commented Dec 13, 2019 at 16:27

1 Answer 1

Reset to default
0
$\begingroup$

I used multiple proportion (or mean) testing.

I just calculated the test statistics between all the groups using t-test. This test takes care about the number of observations per sample (degrees of freedom) when deriving a distribution of the statistics.

After that I used significance level (alpha) and adjusted it to the number of hypothesis tests that I had run (following Bonferonni methodology).

Then I made inferences. By using many t-test (all-to-all) you are explicitly able to say which exactly group differs from which. It is best I think.

If you encode your groups as dummy binary variables and feed them to a linear logistic model, you can just say which input variables are significant predictors of the customer property (buy/redeem), but you cannot immediately say if one input variable is a stronger factor than the other. It is maybe more appropriate to try ANOVA with contrasting on inclusion of different variables into a model.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Can you explain how you are using t tests with a binomial dependent variable? or summary counts? $\endgroup$
    – Sal Mangiafico
    Commented Dec 13, 2019 at 16:29
  • $\begingroup$ @SalMangiafico, that was quite a while... to disambiguate, I am not sure if it was t- or the z-test. I think the approach was following a Gaussian approximation logic: a) I used the binomial distribution moments caluclated explicitly (taken from here en.wikipedia.org/wiki/Binomial_distribution); b) I used the formulas for SE and statistic explicitly (stattrek.com/hypothesis-test/…), (you can go likewise with the t-test: stattrek.com/hypothesis-test/…); c) I got p-values from the stats::qt. $\endgroup$
    – alexeymosco
    Commented Dec 18, 2019 at 10:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.