Pooling data across experiments- correlation

Question

My question is basically about a procedure introduced in a statistical web-site (http://labstats.net/articles/combining_experiments.html), which is about combining data from multiple studies. In this web-site, one of the procedures introduced was to first conduct ANCOVA to examine whether regression slopes are different across experiments. If not significantly different, the author proposes that pooling all the data into a single regression analysis is acceptable (see the section, “Combine all the data into a single analysis” in above web-site). It was the case for my data. I feel this procedure seems reasonable; however, I would like to have insights from wider view. Here are the details of my data: I conducted 5 psychological experiments to examine relationship between pleasantness of stimuli and physiological responses (level of stress hormone) to the stimuli. Different sets of stimuli are used across experiments. Subjects are not overlapping (none participated more than two exp.). In each experiment, I found significant negative correlations between pleasantness ratings and level of stress response (when stimulus is more unpleasant, stress response was higher). In addition to reporting experiment-wise results, I would also like to present results across experiments to summarize the study.

Answer 1

I think you are re-inventing individual participant data meta-analysis. You have all the data from the five primary studies so you can analyse it as though it were one big study. You would probably want to include a fixed effect for study with five levels and then possibly include an interaction between it and any of the moderator variables which was of interest to you to check whether the effects did indeed differ between studies. I would not advocate doing a preliminary screening analysis to check for differences as then all you results are conditional on your having made that decision which messes up your inferences. One thing which worries me is your statement that you do not have overlap followed by stating that some people took part in two studies. That, for me, is definitely overlap and you probably need to incorporate a random effect for participant in your model.