# Design of experiment in multiple regression

Suppose we have the following model of our environment:

$$\hat{y}_t = e^{dayofweekeffect} * x_{1, t}^{\beta_0} * x_{2, t}^{\beta_1}$$

which we can linearize into: $$log(\hat{y}_t)= dayofweekeffect + \beta_0 * log(x_{1, t}) + \beta_1 * log(x_{2, t})$$

To imagine this, lets assume that $$y$$ consists of daily sales and $$x_{1, t}, x_{2, t}$$ different daily investments, lets assume promotional activitiy expenditure. Our investments are not the sole driver behind sales, the baseline sales is modelled by $$dayofweekeffect$$. We can imagine that the $$dayofweekeffect$$ is modelled by fourier terms or dummies. Notice that we dont "control" for the dayofweekeffect in the sense that we have an proxy for dayofweekeffect but instead we use dummies or fourier terms and models it "simultaneously". We could also imagine some structural time series model trying to decompose the effects.

The dataset is problematic since the previous analyst, did not consider the baseline sales but instead used an model $$log(\hat{y}_t) = \beta_{dayofweekeffect_0} * log(x_{1, t}) + \beta_{dayofweekeffect_1} * log(x_{2, t})$$ and used this model to optimize expenditure.

This means that he essentially created datapoints for $$x_{1, t}$$ in which he spent less when the $$dayofweekeffect$$ was lower and more when the $$dayofweekeffect$$ was higher.

My goal is to design an experiment(DOE) such that we can efficiently disentangle the $$dayofweekeffect$$ from the effect of $$x_{1, t}$$ but i need some guidance on which literature/design criterions i should further investigate.

Question What would be an appropiate design criteria for this scenario?

Also notice, that we are not sure how well he optimized the previous expenditure, there might not be an perfect collinearity between the unknown $$dayofweek$$-demand and $$x_{1, t}$$ thus it could be useful to also factor in the previous dataset in some way.

• It would help if you were to explain investments, i.e., are these stock market equity positions? Marketing factors such as advtg, price changes, promotions, etc.? Either way, imo, baselines are not an exogenous factor. Rather, they are a tbd outcome sometimes modeled with a lag of daily sales expressing the inertia of same. Wrt regression model designs, panel data models are common, e.g., Pesaran Time Series and Panel Data Econometrics. Another one is en.wikipedia.org/wiki/Regression_discontinuity_design Sep 25 at 10:23
• updated to promotional activity, notice that the question is about DOE(design of experiments) namely choosing future datapoints such that we can disentangle the effect(in this case) - will edit it further. Certain optimality crtierions such as D-optimality, E-optimality etc are of interest from this perspective, but i might have misinterpreted your comment.
– hugh
Sep 25 at 10:39
• Another question, what is the unit of analysis? The market as defined by an entity such as Nielsen? The store? The household? Among the best marketing science references is Hanssen's book google.com/books/edition/Market_Response_Models/… Sep 25 at 10:47
• the store, will take a look at the book
– hugh
Sep 25 at 10:51
• also, lets assume that there is no brand-building coming from our $x$-variables, thus they dont affect the baseline. One obvious solution is to completely randomize our $x$-variables, but this would probably be quite ineffective, one should be able to utilize the prior information and previous spending to come up with something smarter when constructing the future experiments(allocations)
– hugh
Sep 25 at 10:59

I would randomly split your stores (or groups of stores if they are close enough to worry about spillovers) into three groups: marketing spend as usual, reduce spend, and increase spend. If you have two marketing channels, then you would have 9 groups. This can keep your marketing spend roughly constant while allowing you to learn about the average effectiveness of the change in spend. You can also take one group to zero if you want to estimate a zero-spend baseline. You can also do a two-cell experiment if that more closely matches your goal. You can also vary spend more continuously, but that is often tough to do from a setup perspective and may not work well if there are carry-over effects from day to day. The advantage is that it lets you trace out a revenue-spend curve/surface instead of just 3 points on it.

Then run a regression like:

$$sales_{it}=\alpha + \gamma_{Mon} + \gamma_{Tue} + ... \gamma_{Sat} + \delta_{Up} + \delta_{Down} + \eta_{Mon \cdot Up} + \eta_{Mon \cdot Down} +... + \eta_{Sat \cdot Up} + \eta_{Sat \cdot Down} + \varepsilon_{it}$$

This data is at store and day level. The omitted group is Sunday and normal spend, so all the effects are relative to that. You can even add a store fixed effect to pick up differences between store trade areas that are constant over time.

If you had two channels, you would have more terms and can also include interactions between channels.

You can log sales or just use a Poisson model with robust standard errors. If you randomize over groups of stores, you should cluster your SEs by group.

You can do minimum reliably detectable effects using simulation on historical data. This should work well unless you have strong seasonality and are going into a slow period. Larger changes should be easier to detect.

• i should have mentioned that we consider a single store and not a set of stores, i should also have mentioned that the loglog model is for capturing saturation and synergies. My main interest was really in planning future expenditure and not parametrization. I should have been more clear about these things, i'll create a new thread which really focuses on the different design-criterions like D-optimality, E-optimality etc. However, i'll accept this answer as it is great given the question as it is written.
– hugh
Sep 28 at 20:32
• You can certainly log the outcome and the continuous spend variables or do some sort of hybrid of Poisson with logged Xs to get saturation. Assuming the variation in spend is experimental, then you can interpret the coefficients on DOW x spend and on spend as elasiticities (or calculate them in the case of Poisson). You can use these estimates relationship to plan future spend. Sep 28 at 22:39
• yep, no ideas on which design-criterions would be appropiate in this scenario? D-optimality seems interesting as it essentially breaks multicollinearity
– hugh
Sep 29 at 6:55