0
$\begingroup$

I am trying to find out what statistical tests to run on a set of aggregate data (no individual data) on 3 different groups, variables are summarized below: Group 1 in site 1: Mean value (blood sugar) of the total sample before treatment, and mean value (blood sugar) of the same group after treatment Group 2 in site 2: same as above Group 3 in site 3: same as above

I want to know if it makes sense to run the following analysis on the above data:

1) Paired t-test on each group separately for the group difference in blood sugar means after treatment and before treatment (to test if any increase or decrease in the whole group average is statistically significant) 2) where I start struggling is when I think of the tests to apply for the difference between the groups. I can calculate absolute difference in the means after and before treatment for each group, have then difference of the means after and before in group1, difference of the means after and before in group 2 and same for group 3. Can I run an ANOVA on the difference means between the three groups (mean diff of means after and before group 1 vs mean difference of the means after and before for group 2 vs the one of group 3)?? If yes what other data do I need?

$\endgroup$
2
  • $\begingroup$ Can you clarify - you only have 6 data points (numbers) which are the means from a larger dataset? Why aren't you using all of the data? $\endgroup$ Mar 6, 2017 at 20:19
  • $\begingroup$ Hi Groovy, there is around 1000 data points per site (the 6 I have put was only to give an example), each data point represents the actual test values per individual before and after treatment. Unfortunately I do not have access to this raw data. I only have access to the means of this 1000 data points per site at two point in time, before and after treatment (eg to be more clear: for site 1, I have mean1 of test values for all the 1000 individuals before treatment; and the mean2 for all the 1000 individuals after treatment - It goes the same for the 3 sites). Does it make it more clear? $\endgroup$
    – Pamela
    Mar 7, 2017 at 9:01

1 Answer 1

2
$\begingroup$

I have edited my answer to correct a mistake.

Use ONE paired t-test to see whether the treatment had any effect on mean blood sugar across the three sites. You cannot test whether there are differences among the sites.

Here is some example data so you can check your calculations against mine (if you need to).

--------------Before   After
Site 1         0.23     0.31
Site 2         0.05     0.11
Site 3         0.17     0.24

The outputs from the t-test are:

  • t-statistic = 12.124
  • degrees of freedom = 2
  • p-value = 0.006734

So, for this data we would say there is a statistically significant effect of treatment.

Aggregated data has much less information than the raw data. This means you can answer fewer questions with it. For example, you will not be able to test whether the treatment has different effects at the different sites (this is known as an interaction effect). The aggregated data has eliminated the replication (the 1000 patients/subjects) within sites that would have allowed you to do this.

For this reason you also cannot test whether there are any differences among the sites. Although you have two values for each site, they are not independent of each other, and it's equivalent to saying you only have one value per site. In general, you need replication (more than one independent value per site) to test for differences between groups.

In your research, you may come across two-way ANOVA without replication. It is used for the situation where there is only one value in each combination of factor levels (Site1 Before, Site1 After, Site2 Before ...) just like in the example data above. It will test whether there is an effect of treatment and also test whether the sites are different. But you must not use this! It assumes that the values in the before and after columns are independent of one another (from different groups of people, not the same group of people). The p-values will be wrong, and therefore any conclusions you make will also be wrong!

In my original answer I suggested that using pearson's correlation coefficient would be suitable to test for differences between sites because it deals with paired data points. But this also suffers from elevated Type I errors (unreliable p-values). There is no way to know whether any differences between sites are due to random sampling error or whether they represent additional site to site variation.

$\endgroup$
13
  • $\begingroup$ Thanks a lot for your detailed answer... it is super helpful. $\endgroup$
    – Pamela
    Mar 8, 2017 at 11:40
  • $\begingroup$ I am pleased I was able to provide something useful. Could you help the rest of the community by "accepting" my answer? Thanks. $\endgroup$ Mar 8, 2017 at 11:44
  • $\begingroup$ Of course! I am happy to share.. Can I please ask for one clarification though? $\endgroup$
    – Pamela
    Mar 8, 2017 at 11:47
  • $\begingroup$ I just want to make sure I got the first part right (and it has to do with my brain being off stats for a while!). What you are actually suggesting to do is to run 3 separate paired t-test for each site independently to assess if the changed in the blood sugar level before and after treatment at each site is statistically significant? $\endgroup$
    – Pamela
    Mar 8, 2017 at 11:48
  • $\begingroup$ Which means a first paired t-test for site 1 (mean site 1 before vs mean site 1 after) that will return one t-test value and a p-value that will allow me to conclude if there was a statistically significant change in the blood sugar level at site 1 after treatment, and then repeat the same for each other site?? What is actually not clear to me from your answer as a conclusion is the one t-statistic for across sites - how does it translate? Sorry if I seem to be a bad student! $\endgroup$
    – Pamela
    Mar 8, 2017 at 11:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.