How to statistically determine if there was sales lift?

Question

Say my company is about to make some changes that they expect will increase sales and customer acquisition in current markets, what would be the best way statistically to determine if that lift was realized or if sales grew at the current rate?

I have two types of data I am looking to do this for, both monthly:

1. Customer acquisition - basically white noise
2. Sales volume - typically follows and upward, linear trend

For customer acquisition I was thinking of sticking with something simple like comparing the monthly average post event to the current monthly average with a t-test.

For sales volume I am not sure what the best approach would be since there is a trend in the data and I am not just looking for a simple level shift. My initial thought is to find the average monthly increase before and after and use a t-test but I am not sure if there is a better way.

Answer 1

To test if a recent value or a set of recent values are "different from what was expected" ... I would construct a hybrid model using needed ARIMA structure and user-suggested causals and any empirically found Intervention Variables via Intervention Detection http://docplayer.net/12080848-Outliers-level-shifts-and-variance-changes-in-time-series.html using the historical data . If user specified causals are in play then this would be a Transfer Function . See Time Series Forecast / Transfer Function for an example.

I would then add a dummy variable to the model which was based upon the historical data . I i had 100 observations in the past and I wished to test whether the 101'st observation was exceptional that would be series of 100 zeroes followed by a 1. The test for the new period effect would be based on the t value for this hand-crafted dummy indicator.

In a similar fashion, if I wished to test whether a set of new values deviated from expectations the one handcrafted dummy series would have a set of '1' values not just a single "1'.

An equally viable 1 step approach would be to use all the data ( say 101) values and investigate whether or not a pulse is found at the last point ( i.e. the 101'st value ). If it is found then one can conclude that there was an exceptional activity at that point.

Confidence limits around a prediction might seem like a good approach but unless monte-carlo resampling (bootstrapping) is used symmetry assumptions are not normally ( joke here ! ) correct or useful.