Which statistical test is most appropriate for these data?

Question

I'm trying to design a statistical test that will confirm whether an event x causes another event y. Or more plainly, whether a specific marketing action actually increases telephone calls from potential customers.

Let's say there's a list of data like this:

The data contain around 100 data points, each recording a telephone call event (y), the date of the call, and the days since the last marketing event (x).

The set of hypotheses is as follows:

H0: Regardless of days since last marketing event, calls do not occur more frequently.

H1: Calls occur most frequently between 2 and 4 days since the last marketing event. 

In the full data set (+100 data points) the number of data points in the group (Let's call it group A: 2<=x<=4) is larger than that of the group (group B: 2>x>4). But I need a test to help me be sure that the difference is statistically significant.

I mean I could just look at the two sums of each group (ex. A=65, B=44) but of course that's not convincing or appropriate.

What test would best fit these data?

Updates / Clarifications:

• 'days_since_marketing_event' can be any whole number.
• the core question I'm trying to answer is does x cause y.

Answer 1

In the list of data points you provide I see the following:

• There are $N=14$ observations.

• Each observation of your variable is a single integer value.

• The empirical probability mass function of your variable days since last marketing call (let's call that variable $x$, Ok? Ok.) is:

  • $P(x=0) = 2/14$
  • $P(x=1) = 1/14$
  • $P(x=2) = 6/14$
  • $P(x=3) = 3/14$
  • $P(x=4) = 2/14$

Now let's look at your null and alternative hypotheses for a moment. You wrote:

$H_0$: Regardless of days since last marketing event, calls do not occur more frequently.

and

$H_{1}$: Calls occur most frequently between 2 and 4 days since the last marketing event.

Typically, when performing frequentist hypothesis tests the alternative hypothesis represents the complement (or negation) of the null… that is, if the null hypothesis says $H_{0}$: The world looks like this, then the alternative hypothesis would be $H_{1}$: It is not the case that the world looks like this. $H_{1}$ is literally just a negation (complement) of $H_{0}$. So I think your alternative hypothesis could be rewritten with that in mind.

But before we rewrite $H_{1}$, let's look at your null hypothesis, which has, if your will pardon my saying so, a funky complex grammatical construction:

• You begin with a dependent clause Regardless of days since last marketing event

• You conclude with an independent clause that is an incomplete comparison: calls do not occur more frequently ("more frequently" than what?)

Let me suggest (and feel free to write in comments and correct me if I am wrong) that you are trying to express the simple idea no value for days since last marketing call ($x$) is more or less likely than any other value, or put even more simply $x$ has a discrete uniform distribution. This would imply that (assuming $x$ can take only values between 0 and 4, inclusive) that $P(x=2) = \frac{1}{5} = 0.2$ because there are five equally likely values. There are a few ways you could pose a null and alternative hypothesis around this view of your data:

$H_{0}: P(x=2)=0.2$

$H_{1}: P(x=2) \ne 0.2$

In order to reject $H_{0}$ or not, you would need to answer the question Assuming $x$ is distributed discrete uniform from 0 to 4, how likely is it that I would have observed $x=2$ when $N=14$? The binomial distribution can help us with the arithmetic: under $H_{0}$ the probability of observing 6 data points where $x=2$ out of 14 observations equals $\frac{14!}{6!(14-6)!}0.2^{6}(1-.2)^{14-6} = 0.0323$. If that probability is smaller than your willingness to make a Type I error (i.e. smaller than your $\alpha$, then you would reject $H_{0}$ and conclude that $P(x=2) \ne 0.2$, and therefore your data are unlikely to be discretely uniformly distributed at the $\alpha$ level.

Of course, my reframing of your null hypotheses may not be quite right. For example, perhaps values of $x>4$ are possible? Or perhaps you are interested in testing your empirical PMF versus a theoretical discrete uniform PMF (the one-sample Kolmogorov-Smirnof test may prove useful here)?

Critically: what is the specific question about $x$ you want to answer?

Answer 2

You could do a chi-square test of goodness of fit. This is explained on this page: http://stattrek.org/chi-square-test/goodness-of-fit.aspx?Tutorial=AP

So, let's see what your expected value is. There's always 0 days after a marketing event, but there may seldom be 5 days after a marketing event (if the marketing events often occur 3 days apart). So, we might end up with an uneven number of possible days since event. The number of days can be percentaged and applied to the data so we get an expected number of the 102 calls (I wanted to use a number different than 100, for clarity).

Since the days data are ordinal (4 days is longer than 3 days, etc.) you could also use a Kolmogorov-Smirnov one-sample test.

Answer 3

Categorizing your data (2<=x<=4) needs some theory to support it. Is there a marketing theory that says calls in this region are optimal? If so, you could do a multiple regression analysis with planned contrasts.

Answer 4

I ended up reshaping the data set into that seen below. I realized that there was a bit of redundancy in these data -- repeated information. So on a whim I tried combining the set of data points occurring on the same day into a single data point. This yielded a new variable, essentially a count of the previous data points.

In other words, if six call events happened on a given day, they were then combined into a single data point containing a new variable 'call_count' equal to 6.

After reshaping the data, I selected two samples from it. One sample (x) where the days_since_marketing_event was >= 2 and <= 4, and the other (y) where the days_since_marketing_event was all other numbers (ie. either < 2 or > 4). Note the sample sizes differed.

Based on feedback from other users, I also revised my hypotheses to the following:

I chose to perform a t-test and was unable to reject the null hypothesis.

Though I ended up going with a different solution, thanks to everyone who posted the other interesting and helpful answers.