# Analysis with unequal starting groups

Background: Our business wishes to perform an demand analysis based on two marketing treatments A and B. Due to constraints imposed from the onset, there will be a 50/50 split (receive/not-receive) for treatment A, but a 90/10 split for treatment B. This results in a 2x2 treatment matrix as such:

      B    noB
A   45%  5%
noA   45%  5%


We used a randomizing method to assign customers to each group, following the distribution outlined above. Before commencing our test, we checked a basic parameter (email addr. per cust) and found that within the B/noB group there is a significant (by GLM) difference. No sig difference is found between quadrants or between A/noA. Reminder, this was just a check to see if we are starting with equivalent groups.

Question: Because we already know that there is variation in our groups, and assuming we are bound to the unequal distribution of customers to treatment group, are we doomed in terms of statistical analysis after we run our test? Are there precautions we can take before running the test? Should we re-sample our groups using another method? Any advice would be greatly appreciated, even if it is pointing to another thread or academic article.

Please let me know if this is unclear...

Thanks!

• How confident are you that this parameter has an impact on the outcome of the test ? If I understand it right, you have not run it yet, right ? May 26, 2015 at 20:11
• Pretty confident based on historical data. One thing I didn't make clear earlier, is that multiple custs can share an email, but the constraints of the test prevent two customers who share an email from being in two different treatment groups. Customers can also have multiple email addresses that are share (or not) with others. This means that when we randomize into groups, we have to also pull in the 'network' of customers created by email sharing chains. It is this issue that introduces variation into our samples. The error bleeds into other variables such as historical performance. May 27, 2015 at 19:36