6
$\begingroup$

I am having some difficulty understanding the difference between and identifying an observational vs quasi-experimental design. From my understanding, an observational study is one in which the researcher does not influence the system and only records what they observe (duh). In an experimental study, the researcher manipulates the experimental units such they have different treatments and measures some resultant metric(s), usually to compare them. My understanding of quasi experimental studies is that the researcher uses groups that are already different from one another rather than apply the treatment to the EUs themselves.

For example, let's say a researcher is making observations on bird diversity in different land use types (eg forest vs agriculture). Odds are he or she isn't going to make a forest and farm and put birds in them to see which survive. They are going to go to several farms and several forests and observe the birds that live in each.

Now, it seems that this is a quasi-experimental design based on my above definitions, but does that mean that every comparison study is going to be either experimental or quasi experimental? I can't think of particularly many studies that would fall under the observational category, in that case (correlational, descriptive).

$\endgroup$
  • $\begingroup$ Check out the answers. Do any of them answer the question? $\endgroup$ – Dr. Beeblebrox Sep 24 '15 at 8:01
4
$\begingroup$

I can try to give an example from my own field, econometrics:

Economists are interested in "returns to schooling", i.e., how much more do you earn per additional year of schooling obtained.

An experiment is not an option, as, for good and obvious reasons, you cannot force people to continue or stop their educational career just because of the empirical analysis.

Observational data is generally tricky to interpret if you are interested in the causal effect of another year of schooling, because there are "confounders" that imply that the (generally positive) correlation between schooling and earnings is not (fully) causal. For example, more able, motivated and careful individuals can be thought to choose to obtain more schooling, and such individuals would have at least partly done well in the labor market without additional schooling.

Now, sometimes nature or law is kind enough to hand you a "quasi-experiment". In the above example, researchers have for example exploited changes in compulsory schooling laws. If, effective say 1955, all students in a country are obliged to attend secondary school for, say, 10 rather than 8 years, there will be at least some students who obtain more schooling only because of the new law, and not because they choose so.

Instrumental variable approaches may then be a credible way to exploit this so-called exogenous variation to say something about the causal effect of schooling.

$\endgroup$
3
$\begingroup$

First, as far as you have described the research design, the study is not a quasi-experiment.

I prefer the term natural experiment to quasi-experiment, because I think it more clearly communicates the fact that treatment needs to have been randomly assigned (or as-if randomly assigned). I use the term natural experiments below, but I consider the two equivalent in meaning.

You are correct that experiments are confined to those situations where a researcher actually manipulates treatment assignment.

Observational studies comprise anything that was not an experiment. Natural experiments are a subset of observational studies, but in a natural experiment units were assigned to treatment in a random process (or as-if random, or almost random).

You might look for a natural experiment (or quasi-experiment) if you were seeking to identify the causal effect of a treatment on a set of outcomes. Then you would look for a situation where assignment to that treatment was assigned randomly (or as-if randomly) by nature or a government program, for example. For example, if you wanted to study the impact of forest fires on bird diversity, you might find a place where the government has defined that it will fight fires when they come with X miles of residential areas. After forest fires, you could compare (i) bird diversity in areas affected by the forest fire just a little further than X miles away from residential areas (treatment group) to (ii) bird diversity in areas just a little less than X miles away from residential areas (control group). Because birds would not choose where to live prior to fire based on the government's designation of the distance X, we can expect that before the fire on either side of the X-mile cutoff, birds would be identical on average. There assignment to treatment (being "treated" by the forest fire) is as-if random on either side of the X-mile cutoff. This design is called a regression-discontinuity design [1] or a geographic regression discontinuity design [2].

Also, see more discussion of the difference here: Panel study is a quasi-experimental study? Quasi-experimental is the same as correlational?

  1. https://en.wikipedia.org/wiki/Regression_discontinuity_design
  2. "Geographic boundaries as regression discontinuities." LJ Keele, R Titiunik. Political Analysis, 2014
$\endgroup$
1
$\begingroup$

I would like to answer your question from the Epidemiology point of view.

Basically,there are three kind of studies in Epidemiology, observational study, Experimental and theoretical study.

For observational studies, as a researcher you will not give any interventions to any groups you will study. You just collect data cross-sectionally, retrospectively or prospectively.

For experimental design, as a researcher you will allocate your intervention to some groups and other groups will not receive your intervention.

There are randomized experiments (such as clinical trials) and non-randomized experiment.

For randomized experiments patient belongs to which group is determined by randomization procedures).

For non-randomized experiment, which is also called quasi-experiment, there are no randomization procedures to allocate patients to different groups, it might just be done by convenience.

$\endgroup$
1
$\begingroup$

The quasi experimental design is the one that uses an "experimental research procedure" but all not extraneous variables are controlled. Quasi experimental designs lacks of random assignment of participants to groups. Only in strong experimental designs this is achieved.

Causal inferences can only be done from quasi-experimental designs if (1) cause and effect covary, (2) cause must precede effect and (3) rival hypothesis must be implausible (so the relationship between variables must not be due a confounding extraneous variable).

Now, the third condition is hard to achieve since there is no randomization.

So we can see Quasi-experimental designs to a better option than weak experimental designs and not as good as strong experimental designs.

For your bird-watching scenario, using a quasi-experimental design will not have a conclusive result and the relationship between the type of bird and the land use parameters might be affected by other variables like weather, migration seasons, temperature, humidity, wind orientation, etc. However this might be good enough for your study if you are not able to apply an strong experimental design.

On the other hand, the observational study draws inferences from a sample to a population where the independent variable is not under the control of the researcher. The observational study is then more into the data collection process, where you as a research must collect what you can, to draw inferences from there. The inference in this case (statistically speaking) could be managed by the amount and quality of attributes recorded. naturalistic observation is conducted in real world observations and subject to noise and error. Since the observational study might be conducted in a single farm or a couple selected from the near-by surroundings, this will not be using a randomized sample which might also be prone to statistical error for causal relationships. The only way the observational study will be good enough to demonstrate cause and effect, will be when it is ran under laboratory conditions, say your birds are in a controlled environment, where several domes are created that represents each land/farm type and then you basically observe behavior or whatever you are measuring. The laboratory observation is closely similar to a quasi-experimental design since you are having control of a variable (the setting).

hope this helps.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.