What statistical model should I use to analyze the likelihood that a single event influenced longitudinal data

Question

I am trying to find a formula, method, or model to use to analyze the likelihood that a specific event influenced some longitudinal data. I am having difficultly figuring out what to search for on Google.

Here is an example scenario:

Image you own a business that has an average of 100 walk-in customers every day. One day, you decide you want to increase the number of walk-in customers arriving at your store each day, so you pull a crazy stunt outside your store to get attention. Over the next week, you see on average 125 customers a day.

Over the next few months, you again decide that you want to get some more business, and perhaps sustain it a bit longer, so you try some other random things to get more customers in your store. Unfortunately, you are not the best marketer, and some of your tactics have little or no effect, and others even have a negative impact.

What methodology could I use to determine the probability that any one individual event positively or negatively impacted the number of walk-in customers? I am fully aware that correlation does not necessarily equal causation, but what methods could I use to determine the likely increase or decrease in your business's daily walk in client's following a specific event?

I am not interested in analyzing whether or not there is a correlation between your attempts to increase the number of walk-in customers, but rather whether or not any one single event, independent of all others, was impactful.

I realize that this example is rather contrived and simplistic, so I will also give you a brief description of the actual data that I am using:

I am attempting to determine the impact that a particular marketing agency has on their client's website when they publish new content, perform social media campaigns, etc. For any one specific agency, they may have anywhere from 1 to 500 clients. Each client has websites ranging in size from 5 pages to well over 1 million. Over the course of the past 5 year, each agency has annotated all of their work for each client, including the type of work that was done, the number of webpages on a website that were influenced, the number of hours spent, etc.

Using the above data, which I have assembled into a data warehouse (placed into a bunch of star/snowflake schemas), I need to determine how likely it was that any one piece of work (any one event in time) had an impact on the traffic hitting any/all pages influenced by a specific piece of work. I have created models for 40 different types of content that are found on a website that describes the typical traffic pattern a page with said content type might experience from launch date until present. Normalized relative to the appropriate model, I need to determine the highest and lowest number of increased or decreased visitors a specific page received as the result of a specific piece of work.

While I have experience with basic data analysis (linear and multiple regression, correlation, etc), I am at a loss for how to approach solving this problem. Whereas in the past I have typically analyzed data with multiple measurements for a given axis (for example temperature vs thirst vs animal and determined the impact on thirst that increased temperate has across animals), I feel that above, I am attempting to analyze the impact of a single event at some point in time for a non-linear, but predictable (or at least model-able), longitudinal dataset. I am stumped :(

Any help, tips, pointers, recommendations, or directions would be extremely helpful and I would be eternally grateful!!!

Thank you

Answer 1

It is kind of hard to understand what you are trying to do. I see the following questions/problems described:

1. "What methodology could I use to determine the probability that any one individual event positively or negatively impacted the number of walk-in customers?"

2. "I am attempting to determine the impact that a particular marketing agency has on their client's website when they publish new content, perform social media campaigns, etc."

3. "I need to determine how likely it was that any one piece of work (any one event in time) had an impact on the traffic hitting any/all pages influenced by a specific piece of work."

I think you need to work on getting the question you want to have answered clear in your head. Once you have done that, the rest will follow.

I have the following questions:

• Who are your clients? The marketing agencies or their clients? Do the marketing agencies want help in determining which one of their treatments is effective or do their clients want help in which marketing agency to use?

• Do your clients mind if their data is combined with that of their competitors (other marketing agencies)?

• What kind of output is your model supposed to give? A probability, an effect size, ... ?

I hope this helps.