I am having a hard time with what seems to be a very basic research design issue.

The study has 3 variables:

  • Self-control and adjustment are the main ones, since they were measured in pre and post tests, before and after a year long social and emotional program in two different high schools. (I guess this part of the design would be quasi-experimental)
  • The third variable is values in interpersonal relationships. This was measured only one time at the beginning of the program since it tends to be more permanent. Also, the program wasn't designed to have an effect on this, except values can also affect the adjustment and self control of the students. The results show the groups tend to the individualist values vs. collectivistic ones (descriptive design).

I wanted to add hypothesis about this last variable to my paper, but a teacher told me it would be better not to include those, since those are more descriptive hypotheses.

  • What is the best and correct way to include all this information on the paper, so it´s statistically sound?

You can think of the difference between the pre- and post-treatment estimates in a quasi-experimental way if you assume that treatment (the program in question) was applied in an "as-if" random way; that is, students who would most benefit from the program were not more or less likely to be included in the program. By taking the difference, each student acts as his own control. So if you take the difference between the pre- and post-treatment difference in measurements, you can get an estimate of the effect of the treatment---the causal effect of treatment. This is known as a difference-in-difference research design.

I don't exactly understand the second part. You want to show how people in the program had different pre-program measures of interpersonal relationships relative to people not in the program? If this fact is true, it may call into question the "as-if" random nature that you claim in the first part; if interpersonal relationships influence how effective the program would be and people with high, say, levels of this measure are more likely to enter into the program, you can get a biased estimate of the causal effect of the program.(without more information on what you are looking at, it's hard for me to write a story about what biases could arise).

In any case, the difference between these two groups on any measure would be a valid statistic. It would be descriptive, rather than causal, but valid nonetheless.

For a concrete example, I could calculate the difference in incomes for blacks and whites in the U.S. I can do a statistical test to see if this is 0. If it is statistically significantly different from 0, as I suspect that it is, I can reject the null hypothesis that blacks and whites have the same expected income. But this is a descriptive fact, not a causal one. I can only say that the difference exists, not that being black causes the difference. Perhaps blacks tend to have less education and this fact could be the true cause. This is part of the difficulty in searching for discrimination statistically.

There is no reason not to do descriptive analyses; they are perfectly valid, so long as they are not interpreted as causal, and can often provide useful insights.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.