What if, before you begin the data collection for an experiment, you randomly divide your subject pool into two (or more) groups. Before implementing the experimental manipulation you notice the groups are clearly different on one or more variables of potential import. For example, the two (or more) groups have different proportions of subjects by gender or age or educational level, or job experience, etc. What is a reasonable course of action in such a situation? What are the dangers of discarding the original random division of the subject pool and dividing the pool again? For example, are the inferential statistics that you might calculate based on the second set of groups in any way inappropriate due to the discarded first set of groups? For example, if we subscribe to discarding the first division of the subject pool into groups, are we changing the sampling distribution that our statistical test is based on? If so, are we making it easier or harder to find statistical significance? Are the possible dangers involved in repeating the division of subjects greater than the obvious danger of confounding due to group differences in educational level, say?
To make this question more concrete, assume for the sake of this discussion that the topic of the research is teaching method (and we have two teaching methods) and the difference noted between the two groups of subjects is level of formal education, with one group containing proportionally more people with highest educational attainment of high school level or less and the other group containing more people with some college or a college degree. Assume that we are training military recruits in a job that does not exist in the civilian world, so everyone entering that specialty has to learn the job from scratch. Assume, further, that the between group imbalance in previous educational attainment is statistically significant.
Parenthetically note that this question is similar to What if your random sample is clearly not representative?. In a comment there, @stask perceptively noticed that I am a researcher not a surveyor and commented that I might have gotten more relevant answers had I tagged my question differently, including "experiment design" rather than "sampling." (It seems the sampling tag attracts people working with surveys rather than experiments). So the above is basically the same question, in an experimental context.