What statistical analysis is appropriate for a before and after design (where participants are not necessarily the same people) I am trying to figure out what is the best statistical analysis for my data.
I looked at two companies in timepoint 1 (baseline) and timepoint 2.
Company A did an intervention, Company B did not (control).
I am measuring whether the worker's wellbeing has changed from time 1 and time 2. Wellbeing is measured using a survey (5-point likert) with 4 subscales. However, the people who participated in the 2nd survey may not necessarily be the same people who participated in the first. My supervisor thinks that repeated measures should still be used because we are looking at the company as a whole, and we wanted to see if the intervention changed something. There is also unequal sample size between the companies and from time 1 to time 2.
I am also using SPSS.
Thank you very much! Any advice would help.
 A: I think using a weighted hierarchical regression model makes sense. I would look at outcomes aggregated up to the company level because that's ultimately the question: how can we make companies happier. If this were about employees, the study design would have obviously been tooled to follow employees prospectively and also to randomize the intervention within a particular company (so, say half the employees in company A get motivational interviewing and half do not). But that's not how it was done. It was a "community randomized trial". Average the Likert responses for each time period and company as an observation and use the inverse of the variance of responses as the weight. This gives a total of 4 observations. Don't worry about "sample size", since weights account for that.
In an ideal pre/post analysis you would use a model like the following:
\begin{equation}
\mathcal{E}[\mbox{Score | Time, Company}] = \beta_0 + \beta_1 \mbox{Time}  
+ \beta_2 \mbox{Company} + \gamma \mbox{Company} \times \mbox{Time} \end{equation} 
Where each of Time and Company are considered a binary variable: Baseline versus follow-up, Company A versus B. And it is the last coefficient $\gamma$ which measures the difference in differences from baseline to follow-up, accounting for differences between general company profile. The "excess" difference in time in Company A is what we call the intervention effect.
However, the issue of correlated data remains. The extent of correlation in baseline to follow-up is a function of how many people are in both samples. The more people in common, the more correlated the pre-post results are... usually. Uncontrolled correlation like this tends not to lead to biased betas, but incorrect standard errors. This tends to lead to inference that is conservative, but I cannot promise that this is the case.
This is simply a matter of you, the analyst, verifying as much as possible to convince readers about the generalizability of results. Was drop-out of baseline participants relatively equivocal between the two companies? Was the number of new-hires also equivocal in the follow-up? Among people whom individually you could verify pre-and-post follow-up, was the results stemming from a correlated data analysis consistent with the findings from the heirarchical one?
