Problem description: Every 3 weeks a fashion company sends out an expensive booklet with descriptions of clothes to each customer from the company's electronic records. There exists a purchase history what each customer bought in the 3 weeks after receiving the booklet and pricing information about clothes. The problem is to find those customers to whom it makes sense to send the booklet, so that fewer booklets are sent.
Additionally to this data, there is also a small dataset containing information what customers bought in a period of 9 weeks when no booklet was sent out.
I cannot change this data, this is all I got and I need to work with what I have.
A more detailed description can be found here.
I can hypothetically
imagine a variety of different models that output some selection of customers that the model thinks will be customers that will buy lots of things when they receive such a booklet. But the problem is that to train them, for each of these models I need to somehow assess (some variant of) the spending history of those customer that did not receive any booklet, so that I will can use this information as a feature.
Q1: How can I do that assessment, given my fixed dataset?
(The only way I could address this would be to make various assumptions when the booklet had no influence (such as "if product X was not in the past 5 booklets and during this time 70% of all the people bought X, when a customer at current time buys X, it should be considered to be bought as if no booklet had been sent out"), and hope that they were good, since I have almost no way to test them, since my dataset from which I need to come up with a model is fixed.
I was given a hint by seanv507 that the unofficial Google data sicence blog might contain information how I actually might be able to test such assumption; unfortunately the section "Using randomization in training" which is the interesting part telling me how to do that, was a bit too vague and at the same time to technical for me to make much sense of it.)
Q2: Could you let me know if what is in that blog is relevant to my problem - and if yes how to apply it?