I'm about to open the door to a very thorny issue in the social sciences. How does one correctly model and test hypotheses about mediating variables using observational data?
I'm familiar with the Baron-Kenny approach to mediation (see previous answer here), and also with structural equation modeling. However, I've heard both approaches disparaged by more quantitative-minded social scientists than myself -- especially when one is using observational rather than experimental data.
So, let's say that I'm trying to resolve the following:
Y is a behavioral outcome. Both X and Z are observed characteristics of subjects that cannot be manipulated by an experiment. X is an attitude (something that can be changed over the long term) and Z is an unchangeable characteristic such as age, race, etc.
I hypothesize that X is a mediating variable, thus Z affects Y through the pathway of X. While it's reasonable to suggest that Z is in some way correlated with X, my theory argues that it has no impact on Y (other than through **X).
How would one best test these hypotheses using best practices in current research?