2
$\begingroup$

I think the following problem (from Open Intro Statistics by David Diez) describes an observational study, not an experiment. Do you agree?

Researchers studied the relationship between socio-economic class and unethical behavior. 129 University of California Berkeley under-graduates were asked to identify themselves as having low or high social-class by comparing themselves to others with the most (least) money, most (least) education, and most (least) respected jobs. They were also presented with a jar of individually wrapped candies and informed that the candies were for children in a nearby laboratory, but that they could take some if they wanted. After completing some unrelated tasks, participants reported the number of candies they had taken. It was found that those who were identified as upper-class took more candy than others.

  1. The explanatory variable is social class, but isn't this a prospective observational study, not an experiment?
  2. Am I correct that this study cannot establish causality because there is no random assignment to a control group and there is no placebo? What the heck would a placebo even mean in this context?
  3. What I would really like to know is this. If you did observe a correlation between economic class and ethical behavior (let's pretend the design of this study can show that, even though I think their method is silly and unclear), then how would you design a controlled experiment to establish causality between social class and ethical behavior?
  4. For an experiment would you call this an example of stratified sampling? You identify groups of similar economic class and then sample from them? It seems that would satisfy the requirement for randomness. How would you control for the effect in behavior?
$\endgroup$
1
  • $\begingroup$ Aside from going Pearlian on the problem - if you wanted to run an actual experiment, I suspect you could achieve it by manipulating the class identification. Imagine you have participants you've determined to have an approximately equal social class and background somehow, but make some rate themselves against an artificially high points of reference and some against artificially low ones. However, this means you need a decent, objective-ish alternative way of measuring the social class. $\endgroup$
    – jkm
    Feb 7, 2020 at 16:37

1 Answer 1

1
$\begingroup$

With these sorts of experiments there is always the question of external validity. To which extent are associations (and theories about what causes these associations) amongst Berkeley undergraduates willing to take part in experiments transferable to more interesting populations?

Leaving that aside:

  1. It is indeed an observational study with respect to inferring the effect of class on something else. Class is measured (by self identification) and then associated with a decision that is part of the experiment.
  2. The finding of the study is that self identified social class correlates with taking more candies from a jar that will go to children. There is nothing happening in the experiment that helps us narrow down the list of theories as to why there might be such a correlation.
  3. You'd need to engineer a situation where you can compare individuals that are the same except in social class. Then you could infer from differences in their behaviour that it is because of the difference in social class. If the goal of the study is to investigate how one's perception of one's social class affects behaviour, it would be possible to experimentally shift it differently for a treatment and control group by asking people to think about, or expose them to, different comparisons when they assess their social class. If however we want to know whether a truly immutable characteristics is causing something, the best we can do is pretty much just adding as many controls as possible: Make sure the people of different social class whose behaviour we compare are the same in everything we can measure. When that's the case, it's my experience of the experimental literature, at least in economics, that doubters of a claim that an association is causal can typically think of an alternative reasons why there is an association, even controlling for all sorts of things.

  4. As far as I can tell there is no randomisation in this experiment. The goal is to get measurements of two quantities (under clean conditions) you'd typically not get to observe: Self identified social class, a certain sort of ethical behaviour pattern.

$\endgroup$
2
  • $\begingroup$ I appreciate your response. I'm curious about something you brought up in #3. I think you don't have to get individuals who are the same except in social class. Isn't this the point about random sampling and controls? If you control for one variable, then other differences tend to distribute similarly in all groups and shake out in the analysis. $\endgroup$ Feb 8, 2020 at 14:46
  • $\begingroup$ Randomisation ensures it in expectation, but ensuring important predictors of the outcome are actually equally split is more efficient. If you want a detailed analysis, I can recommend this paper: arxiv.org/abs/1607.00698 $\endgroup$
    – CloseToC
    Feb 9, 2020 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.