How to sample transactions to estimate customer-level effects Let's say I am trying to estimate the treatment effect of rude waiters on how often customers frequent a restaurant. To simplify, let's assume that rudeness is binary, observable, and random. I consider the population of all transactions in October, and I see that some customers have eaten multiple meals and others only once. My outcome will be number of meals in November-Dec period.
I would like to construct a customer-level cross section out of my panel data since the estimation method I am using gets complicated with non-independent observations. I have three options available to me:


*

*Grab a random transaction for each customer. Use the waiter from that transaction to determine if a diner was treated or not.

*Aggregate the October data by calculating the # of rude waiters/# of meals for each customer

*For each person, grab a random rude transactions if there was a rude one at any time in October. Otherwise, grab a random transaction. 


Unfortunately (2) is off the table since there are other attributes of the meal that are hard to aggregate.
Method (1) has the problem that 2 people who both dined twice and had rude transactions half of the time might wind up in separate groups, which seems strange since they have the exact same history. 
Method (3) seems the most sensible, but it is giving me an estimate that have the wrong sign, and seem fragile.    
I have three related questions:


*

*Is there anything wrong with (3) from a theoretical sampling perspective? 

*Can I rationalize using (1), since on average the mis-assignment bias might cancel out?

*Can I get my cross section in some other way, perhaps by adding weights of some sort?

 A: I think the problem you noted with method (1) is more subtle: I would expect that people would stop visiting a restaurant after they have been exposed to a rude waiter, so a sequence (0,0,1) is plausible, but (1,0,0) is arguably less plausible (1 is for the rude waiter). If that is the case, this would prevent the mis-alignment bias to cancel out, per your second question, as not all permutations and subsamples of the customer history should be considered as equally likely.
Method (3) is essentially a population-based case-control study, in which your sampling design depends on the outcome. I don't know how much sampling/survey statistics background you have, but try taking a look at Scott (2006) Waksberg lecture. It is certainly not terribly wrong from a theoretical perspective, but you need to know how to analyze this kind of stuff properly. Note that Alastair Scott talks about a logistic regression in which the dependent variable is the outcome you sampled upon; in your application, this would mean modeling the probability that in a given transaction, a waiter will be rude. Your research question is different, though, so most of what the case-control outcome-dependent sampling would do is to produce a substantial variability in pweights.
On your third question, there's been some work on inverse sampling (Rao, Scott and Benhin 2003) and balanced sampling (Tille 2011) as possible ways to reduce the complexity of the problem while retaining some other desirable properties.
