I have a treatment group of size 30 (30 schools in California) that used a math supplemental software. In a simple analysis, I'd like to compare students' average Math growth between our treatment group and a comparable control group. There are many schools in CA that didn't use the software. I'd like the control group to include similar performing schools (their baseline scores be similar to treatment schools with a reasonable margin of error). Also, I'd like the control group sample size be 3 times of my treatment (here 90 schools). There are many many choices of 90 schools out of more than 1000 schools in CA. How would you choose your control group?
14
-
1$\begingroup$ You may say the more information, the better. Use all CA schools and fit an ANCOVA (assuming that the distribution of control and treatment have overlap). I agree, it's good. However, I'd like to show that both group had a very similar average baseline score and variance of scores in the baseline which is not possible if we consider the whole CA specially that the treatment group are more low-performing schools. $\endgroup$– SamJul 6, 2012 at 0:20
-
$\begingroup$ Sorry if this sounds unhelpful, but I've voted to close this question. As it's written, there doesn't figure to be any one correct answer, and I'm not sure the question's about statistics. The best response might well be "read everything you can on research design." But if your intention was to solicit opinions about some specific aspect you have in mind about a design for this situation, please focus in on that--it should make for a more productive Q+A. $\endgroup$– rolando2Jul 6, 2012 at 0:39
-
1$\begingroup$ @rolando2 : thank you for your answer. I'm fairly new to Cross Validated. I was thinking that I could contribute to it. I started contributing to Stackoverflow 2 days ago, and although I did spend a little time, but I got reputation of 113. My first experience was a little disappointing that someone voted me -1 and you think we should close the question. I'm a statistician and have worked in this area for the previous 3 years and based on my experience working with real world data, I think it's an important question. $\endgroup$– SamJul 6, 2012 at 1:12
-
1$\begingroup$ @jbowman: Thank you. I'm aware of the website. By writing this question, I meant to ask a general question that how people think of finding a comparable control group. Suppose that we have a potentially 300 comparable schools, how you would choose 90 of them for your analysis? $\endgroup$– SamJul 6, 2012 at 1:16
-
2$\begingroup$ @Sepehr I disagree with roland2 about the appropriateness of your question here. I think it relates to experimental desing is sues and hence is appropriate for CV. Regarding the efficiency issue that I brought up my implicit assumption is that the two populations have the same variances (or at least the variances are close to being equal). In that case an estimate of mean difference based on a fixed total sample size n will less accurate when the sample sizes are unequal and it gets worse as the imbalance increases. $\endgroup$– Michael R. ChernickJul 6, 2012 at 12:08
|
Show 9 more comments
1 Answer
$\begingroup$
$\endgroup$
If I understand correctly, you want to find optimal balance in covariates between your control and treatment group. If that's so I'd use a matching procedure.
R package Matching has all the functions needed for this, including some tools to assess if balance between groups has been achieved after the matching. Check this paper with details about its use and some examples.
