I administered a front office staff customer service satisfaction survey anonymously to consumers at different mental health clinics. Then the front office staff at these clinics received a course aimed at improving their front office customer service delivery. We plan to administer the same survey again (post- survey) to consumers anonymously at the same clinics to then be able to evaluate change in counts in several different variables (answers to likert scale questions) measuring aspects of consumer front office staff satisfaction following the front office staff customer service training.
EDITED TO REPHRASE THE QUESTION WITH THE FOLLOWING ADDITIONAL INFORMATION: The pre- and post- samples are similar in size; each is about 1/7 of the same total population of consumers who are actively receiving services at these clinics.
The main focus of my original question is the following: Given the above information, is there a difference between the following scenarios in how we can determine whether the training had an effect on change from pre- to post- and in attributing change to the larger population?
Scenario 1: The pre- and the post- samples consist of the same individuals, but they are not matched.
Scenario 2: The pre- and post- samples are completely different individuals, but they come from the same population.
Scenario 3: Let's assume there is 80% overlap of individuals in pre- and post- samples.
In summary, in my experiment, the pre- sample size is 4700, and the post- sample size is likely to be 4600-4800. The size of the whole population of consumers actively receiving services at these clinics is 35,000.
Question: How can we tell whether the training resulted in a significant change in satisfaction ratings in each of the three scenarios? Moreover, does having the same individuals in both samples give us an advantage in determining whether there was a significant change?
Thank you in advance.