Differences in potential explanations between observational and experimental studies Imagine we have a categorical variable, A—for example, whether somebody owns a dog or not—and a quantitative variable, B—for example, how many days a person is sick in a given year. Assume that our measurements are valid and correct and the assumptions for a t-test are met. Imagine we have two studies:


*

*In the first study, we send out a survey to a large random sample of people asking about both A and B, and then use a T-test to measure whether there is a statistically significant difference between dog owners and non-dog owners in how often they get sick.

*In the second study, we have a representative group of volunteers. We randomly assign half of them to have a dog in their home and half not to for the next year, and then we measure how many days they are sick. We then use a T-test to measure whether there is a statistically significant difference between dog owners and non-dog owners in how often they get sick.
In the first study, there are a lot of reasons we might get a statistically significant difference:  


*

*It could be random chance; we'd expect that to happen once in a while.  

*It could be that A causes B, which is what we're looking for.  

*It could also be that B causes A.  

*It could be that some other variable, C, causes A and B.  

*(There might also be other options.)  


Is it correct to say that an experiment showing a statistically significant difference has the same potential explanations as an observational study showing such a statistically significant difference, except for B causing A and C causing A and B? If not, what are the other options and why do they exist for an observational, but not experimental, result?
 A: Yes, it is correct to say that with an experiment, you have ruled out the reverse causal explanation.  And, unless you have other experimental designs that rule out the other plausible alternative explanations, then both the experimental and observational study will have the same plausible alternatives (e.g., same threats to validity).  The arguable benefit for the experimental design is that you have ruled out the reverse causal relationship, a.k.a., the temporal condition.  This brings us one step closer to Hume’s criteria for causality:
(1) A has an observable effect on B
(2) A occurs before B
(3) No other explanations exist.
You can argue (1) for both an observational and experimental study.  You have guaranteed (2) for an experimental study (though it may be true for an observational study, too, but it may be related to a confounding variable).  But, as you suggest in your query, (3) remains unanswered for both designs.
Happy to elaborate further, though a great reference for this would be Campbell & Stanley’s (1963) seminal work on experimental and quasi-experimental designs.  (Note:  they address the issue of randomization vs. representation in this work, though I chose not to elaborate here.)
A: I'm sure you'll get great answers, but will give you a different perspective than what's usually considered on this subject. I claim that it's not just the question of experimental design. It's also the phenomenon itself, its stability.
Consider this: we observe that more Sun in the skies is correlated with warmer climate. For instance, summer time there's more Sun, and it's warmer. In South there's more Sun, and it's warmer etc. You can't really test this directly experimentally. Yet, I doubt you'll find many people contesting this notion these days. 
In this case we have a very stable phenomenon, and the totality of our knowledge of astronomy and physics would rule out any other explanation. Thus my claim is that it's not just about the design of the experiment, it's also other factors such as our prior knowledge and the nature of the phenomenon under study
