0
$\begingroup$

I have a small dataset with a group of members which were exposed to a treatment and then again, their purchasing behavior was measured. Please note that there is no control group here, it is a case of paired data. The goal is to check the effect of treatment on purchasing behavior of the customer. My data has following columns, no. of days with purchases after treatment, no. of days with purchases before treatment, no. of days with non-purchases before treatment, and no. of days with non-purchases after treatment. All this data is available for each member. e.g.

member_id | after_purchase_cnt | before_purchase_cnt | after_no_purchase_cnt | before_no_purchase_cnt 
123       | 4                  |         7           |    8           
|       5

To explain, above member

  • made purchases on 4 days after the treatment
  • made purchases on 7 days before treatment
  • Did not make purchases on 8 days after treatment
  • Did not make purchases on 5 days before treatment

So, the conversion rate is given by 7/12(before the purchase), and 4/12(after the purchase). similarly for thousands of other members.

Since my data (distribution for conversion rate) is highly skewed for both before and after the treatment, it makes sense to go for Fisher Exact test. my question is, since we need to build a contingency table( 2 by 2) for my case, what does the cell contain for fisher test? Will it be okay to put the data in following format:

        No. of Days With Purchases  No. of Days with non-purchases

Before         a                              b

After         c                               d

Here a denotes total number of days when purchases were made by all members in the group before treatment. For example, if

member 1 made purchases on 4 days before treatment, member 2 made purchases on 3 days before treatment.

Then we will have a = 7, provided we have only two members. Similar explanation for others. Can I kindly get some help whether my problem formulation is correct? Or there is an alternative way to make the contingency table. And, may I know how to implement it in R?thanks.

$\endgroup$

1 Answer 1

0
$\begingroup$

In trying to understand your question here, it seems that it is something like a bi-directional cohort? Where each member has data from 12 days before and after? Then your cells are counts of actual days where a purchase was made or not made?

If so, I don't see why Fisher's exact test would be used here, unless you are trying to test what the treatment is doing to each individual person. My assumption is that you may want sum the counts for all the members in your study, in which case the cell counts would be higher than 5, which is generally the cutoff for a Fisher's exact test.

A better method may be to sum all of the counts of days purchased (successes) before and after treatment, and sum the total days (trials) and then do a proportional Z test to see if the proportion of days purchased before is different than after.

If this makes sense, please let me know and I can provide you with R code. If I misunderstood your study design, please rephrase to make it easier to understand.

EDIT: (R code for test of proportions below)

prop.test(x = c("successes_group1","successes_group2"),
n = c("trials_group1","trials_group2"), alternative = c("two.sided"))

where successes_group1 is the number of days purchased before treatment and successes_group2 is the number of days purchased after treatment. Trials_group1 is the sum total number of days subjects had the opportunity to purchase before treatment and trials_group2 is the sum total number of days subjects had the opportunity to purchase after treatment. Two.sided tells the code that you are testing if the proportion of days purchasing can be either increased or decreased, so leave that as is.

As far as distribution, is really doesn't matter. This test is analogous to a chi-square test, but I find it slightly easier to comprehend what it's actually testing.

$\endgroup$
4
  • $\begingroup$ Thanks for replying. Can you kindly let me know, is there a particular reason why Fishers test is not suitable here. We are looking for effect of the treatment on purchasing behavior for each customer. Kindly conform if question is making more sense now. i have added some more details. $\endgroup$
    – jayant
    Commented Dec 23, 2019 at 18:53
  • $\begingroup$ If looking for each customer, I suppose it makes sense, although in the interest of generalizability, I would strongly suggest summing the counts for all subjects and using the proportional Z-test method for three reasons. First, you are increasing your statistical power because you are using all of your data in one test. Second, it deals with the issue of the format of days with purchases and days with non-purchases, as they are measuring the same thing. Third, knowing how a treatment influences an individual's purchasing habits is not useful, as it is not generalizable to more individuals. $\endgroup$ Commented Dec 23, 2019 at 19:11
  • $\begingroup$ Thanks for the explanation.Can you kindly provide the R code to implement Z test for proportion?are there any assumptions to be checked regarding the distribution.thanks $\endgroup$
    – jayant
    Commented Dec 26, 2019 at 2:30
  • $\begingroup$ Check the edit on my original answer. $\endgroup$ Commented Dec 27, 2019 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.