What to do with confounding variables?

I need to do an experiment. First let me describe present situation. The company that I work for is a cinema. It has a gaming section where people who are waiting for movies can pass time by playing games. People can pay only by using prepaid membership card. Unfortunately this gaming section is not generating enough sales. We are trying to find the cause(s).

My hypothesis is if we accept cash as payment, sales will increase.

My plan is to have experimental group and control group. The experimental group will accept cash payment, the control group doesn't. The sales of both groups are tallied before and after the experiment.

The difficult thing about this is that I can't find a way to isolate the 'cash payment' factor from other factors:

• When the movie playing in the cinema is good, more people will come and sales will also increase
• Each cinema only has one gaming section, I can't split it into two sections (one accepts cash, the other doesn't)
• If several sites accept cash and several others don't, I don't think I can compare the results directly because the visitors are different, the number of gaming units are different

I'm looking for suggestions to isolate this 'cash payment' variable, or maybe another approach altogether.

-
How many cinemas are there, roughly? –  onestop Nov 9 '10 at 11:03
Oops sorry, five, one doesn't have gaming section –  Endy Tjahjono Nov 9 '10 at 11:08

Here are some suggestions relating your to bullet points above:

• What about using the daily takings as an explanatory variable?
• What you need to do is form an equation where you predict gaming sales given a number of other factors. There factors will include things you are interested in such as whether they used a prepaid card. However, you need to also include factors that you aren't interested in but have to adjust for, such as daily takings. Obviously, if the film is a blockbuster then gaming sales will increase.
• Suppose you have N cinemas. Select N/2 cinemas and put them in group A and rest go in Group B. Now let Group A be the control group and B the experimental group. If possible, alternate this set-up, i.e. make Group A the experimental setup for a few weeks.
• If you can mix over groups (point above) then this isn't problem. Even if you can't you can include a variable representing the number of gaming units.

The statistical techniques you will probably need is multiple linear regression (MLR). Essentially, you build an equation of the form:

Gaming sales = a0 + a1*Prepaid + a2*Takens + a3*<other things>


where

• a0, a1, a2 are just numbers
• Prepaid is either 0 or 1
• Takens are the daily takens.

MLR will allow you calculate the values of a0-a2. So if a1 is large this indicates that Prepaid is important.

-
I'm not clear about using the daily takings, can you elaborate on that please? –  Endy Tjahjono Nov 9 '10 at 11:05
@endy_c Does that help? –  csgillespie Nov 9 '10 at 11:52
OK so instead of trying to eliminate bias from the movie, I can include it in the experiment, got it, thanks! –  Endy Tjahjono Nov 9 '10 at 12:21

How about comparing the before and after you introduce the cash option across the two groups? Say you assign half the cinemas to the cash option (treatment) and half continue with no-cash (control). Now, you can compare how sales changed in the treatment group following the introduction of the cash option, and also compare how sales changes in the control group. If indeed the cash option is effective, then the change in the treatment group will be bigger than the change in the control group.

I recall reading an interesting statistical analysis done by Prof Ayala Cohen at the Technion's statistical lab for assessing the effect of removing advertising boards from a major highway in Israel on accidents in a similar fashion: to control for other factors that changed during this period, they compared the reduction in accidents before/after to a parallel highway where advertising boards remained there throughout the period.

-
I suppose the treatment cinema and the control cinema should be as similar as possible for this method? They are spread in 3 different cities and may play different movies. –  Endy Tjahjono Nov 10 '10 at 15:23
@endy I am not sure those differences matter. The suggestion is to use change in gaming sales in treatment relative to control and not baseline sales as an indication of treatment effectiveness. Thus, while different cinemas, different movies etc may have different baseline gaming sales, the change in gaming sales would be a function of the presence or absence of the treatment. –  user28 Nov 10 '10 at 15:29
(+1) I was going to suggest something similar. This is sometimes known as a 'difference in differences' estimator en.wikipedia.org/wiki/Difference_in_differences –  onestop Nov 10 '10 at 16:56
Sorry I'm still not clear, the movies are not just different between cinemas, the movies change too. If one cinema starts playing a new interesting movie while the others haven't, this one cinema may have bigger change in gaming sales, no? –  Endy Tjahjono Nov 12 '10 at 1:50

Aside from my practical statistical suggestion, I wanted to raise a slightly different issue: I realize that the cinema's goal is to maximize revenues, and of course the analysis (and strategy) can be geared towards that goal. However, I would like to suggest a broader, holistic view that companies as well as analysts should consider: the overall benefit. In this case, we can consider the value of the gaming addition to cinema goers. Are they happier or more satisfied with the overall experience? (this can be evaluated, e.g., via a quick questionnaire). Or, if the gaming is educational, for instance, then perhaps there is added benefit to those playing? I recall that in several cinemas in the U.S. there are word games on the screen before a movie starts. These can be perceived as fun and educational and could therefore be value added. In fact, if movie goers perceive the gaming service as value added, then they will likely choose this cinema over others and perhaps even visit it more frequently.

What I am trying to say is that it is useful to define "success" in a broad manner and to think big. In the end, success will depend also on the wellness of the "customers" and the impact of "treatments" on society, culture, the environment, etc.

Sorry if this is too philosophical, but I have had so many MBA students maximizing short-term financial gains and too few thinking of issues that are not monetary. Yet, data mining and statistics can be used for broader causes.

-
The preferred approach to answering questions is to provide one answer per question. Ideally, any new material should be integrated with an existing answer. You may want to look at this discussion on the meta site: meta.stats.stackexchange.com/q/635/28 FYI- the meta site is the place to go to if you need to know how the site works, have any questions about the site's functioning etc. –  user28 Nov 11 '10 at 13:56
Thanks for the reminder Galit, we will keep the big picture in mind. –  Endy Tjahjono Nov 12 '10 at 1:14