1
$\begingroup$

I conducted an experiment to assess the effect of five interventions on blood glucose levels in rats. Therefore, the experiment included five groups, with each group containing five rats. Every day and for each rat, I measured the change in blood glucose levels following the administration of the interventions. Therefore, I generated 25 readings daily (change scores). I repeated the experiment 15 times (over 15 consecutive days) and generated 15 sets of 25 readings. The experiments were always conducted on the same rats.

I want to run a one-way ANOVA to assess the differences in blood glucose levels between the five groups. I can set up my data in 2 ways:

  • Calculate the mean change in blood glucose levels for each rat over the 15 days and then run ANOVA. The total sample size in this approach is 25 (5x5).
  • Treat replicates as separate entries and run ANOVA. The total sample size in this approach is 375 (5x5x15).

What considerations should I make when selecting my approach? What are the pitfalls? For example, the second approach is more powerful because of the 15-fold greater sample size, but at what cost? Is the first approach invalid because I am collapsing a lot of data and losing some of the properties of the data?

Improve this question
$\endgroup$
4
  • $\begingroup$ Should use Analysis of covariance (ANCOVA) with random effect or with special variance-covariance matrix. $\endgroup$
    – user158565
    Aug 2, 2019 at 16:22
  • $\begingroup$ Your second approach suffers from a problem sometimes called pseudoreplication $\endgroup$
    – Jake Westfall
    Aug 2, 2019 at 17:19
  • $\begingroup$ @JakeWestfall For a linear model: $Y=X\beta + Z\gamma +\epsilon$, you can specify $Z\gamma$ and or R = Var$(\epsilon)$. My second approach means specify R and give up $Z\gamma$. $\endgroup$
    – user158565
    Aug 2, 2019 at 19:30
  • $\begingroup$ @user158565 I meant the OP's second approach. Your suggestion sounds fine $\endgroup$
    – Jake Westfall
    Aug 2, 2019 at 19:47

2 Answers 2

Reset to default
2
$\begingroup$

I think both approaches are not recommendable. Let's first look at your second approach: Here, you violate an extremely important assumption of ANOVA, namely independence of observations in the rows. The replicates from the same rats are necessarily correlated because they stem from the same rats.

The consequences of this violation are dramatic. The standard errors are too small and you get a lot false positives (see here for an overview). Unfortunately, already minor violations of independence result in substantially increased false positive rates. To cut the long story short: Don't do that if you want to draw any valid conclusions.

Your first approach is in principle valid but it is inefficient because it does not make use of the full structure you have in the data. For instance, it seems reasonable to assume that the trajectory of the Glucose level within each of the 15 days is the same within conditions (or at least within rats). This information cannot be (easily) included in an ANOVA context. This is only one possible constraint that may be interesting.

I would highly recommend you to learn a more sophisticated method, for instance, linear mixed models (https://ourcodingclub.github.io/2017/03/15/mixed-models.html) or latent growth models (https://en.wikipedia.org/wiki/Latent_growth_modeling). Both contain your ANOVA as a special case but offer the possibility to conduct much more informative modeling of your data set.

Improve this answer
$\endgroup$
8
  • $\begingroup$ I checked the assumptions and I fit a linear mixed model (fixed effect: intervention; random effects: day, rat). The approach is unequivocally superior for my data set compared to my original plan. Follow-up question: Had I went for the first approach I mentioned, I could have conducted pairwise comparisons using a post hoc test. What is the corresponding approach in linear mixed modeling if I want to compare interventions pairwise? $\endgroup$
    – Abeont
    Aug 3, 2019 at 18:41
  • $\begingroup$ Could you briefly summarize what you hypothesized about your interventions? There is more than one way to achieve this and it depends on what you want to know. $\endgroup$
    – StoryTeller0815
    Aug 3, 2019 at 19:30
  • $\begingroup$ I would benefit most from running all pairwise comparisons. I understand the most obvious issue with pairwise comparisons is multiple testing and that it may be addressed by several methods (Bonferroni correction, Benjamini–Hochberg procedure, etc...). Interventions A and B are negative controls, intervention C is a positive control, and interventions D and E are under assessment. It would be informative to test if intervention C significantly differs from A and B (successful positive control), A and B do not differ (successful negative controls), D differs from A, B, and C etc... $\endgroup$
    – Abeont
    Aug 3, 2019 at 20:18
  • $\begingroup$ I don't know how it is implemented in the software you use but you could dummy-code your interventions and add the dummies (+ all interactions between them,) to the equation. Than you get build-in pairwise comparisons (though you still have to correct for alpha accumulation). See e.g. here: utstat.toronto.edu/~brunner/oldclass/appliedf11/handouts/… for some more exhaustive examples. $\endgroup$
    – StoryTeller0815
    Aug 3, 2019 at 20:28
  • $\begingroup$ If I understand correctly what you did, you will only compare the mean across all measurements per individual and you will neglect that there are several measurements per day. That is, you don't learn anything about differential trajectories between your interventions. To include this, you would need to add time to the equation. This can also be done by dummy-coding the measurements - or you specify a function for the trajectory of the glucose level. Depends on how much you already know and what you want to learn about the trajectories. $\endgroup$
    – StoryTeller0815
    Aug 3, 2019 at 20:31
1
$\begingroup$

Why did you conduct this experiment on 15 different days, rather than just on day 1 and again on day 15? Are you interested in how the intervention plays out over time? (Is most of the change early on and then steady? Is there not much change at the start and then the importance of intervention becomes clear? Do some rats change immediately, while others take a while? Does the progression differ from group to group?)

Your first method (averages over rats) is sound as far as it goes, but it ignores information on 'progression of effect over time'. If you use this method, you might supplement it by looking at 25 graphs showing changes over time---one graph for each of the 25 rats. Then follow up, if you find anything interesting.

As your description of the second method stands, there is no discussion of how you keep one rat from looking like 15 different rats. Without any particulars, the comment by @user158565 seems to address this issue.

You might do an ANOVA for a partially hierarchical design with three factors: Day $(\delta_i, i=1, \dots, 15),$ Group $(\gamma_j, j = 1, \dots, 5)$ and Rat $(R_{k(j)}, k=1, \dots 5$ within each $j),$ where Greek letters are for fixed effects and Latin for random, and parentheses denote nesting. (Perhaps you'd include a Day-by-Group inteaction.) Other approaches might be an ANCOVA model or a regression model.

For this particular experiment, your intuition and knowledge of the field of study may have led you to an experimental design that will give answers to most or all of your questions. However, it is best to decide on a design and a model for analysis ahead of time. One advantage would be ability to do 'power and sample size' computations that tell you in advance, for example, how many rats you need to use per group in order to have a good chance of detecting effects of intervention that are sufficiently large to be of practical importance.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.