Different (Welch's) ANOVA tests and post hoc contradict paired tests. Which one to accept?

Question

This is a problem that I think nothing would help but sharing my data here. I have 3 groups (n=7) which are paired. They are concentrations of a chemical compound in different parts fish tissue. For example, the 5th number in G1 and G2 and G3 are 3 different measurements from 3 different parts of the same fish. Thus, We examined 3 different parts of each of our 7 different fishes. Shapiro-Wilk and d'Agostino-Pearson tests are significant (p<0.05) but the Levene's test for means indicates the homogeneity of the data (p=0.57). Because of the outliers, I decided to run Welch's ANOVA instead of normal one-way ANOVA. The result is insignificant.

Some other researches on the same type of data, reported the log(10) values and ran ANOVA and observed significance between the groups of their own data. I did so, and the Log-transformation of my data is normal (Shapiro-Wilk Test) and homogeneous (Leven's test).

One way ANOVA (I'm not sure if it is a good idea to use normal ANOVA when n=7 only), is insignificant but when I use contrast post-hoc, there is a significance between G1 and G3. Welch ANOVA and Kruskal-Wallis tests are still insignificant when I use the Log values. Out of curiosity, I performed t-test for independent samples between all of my Log groups and there are significant differences between G1-G3, G2-G3 but G1-G2 is insignificant. Even the same t-tests on the raw data (before logarithms) shows exactly the same results. These results are in accordance with what other researchers have found with their dataset. Before each t-test, I ran a F-test to make sure my variances are not significant.

So, does it mean that I can accept the paired t-test although it is contrary to the omnibus Welch's ANOVA and other tests?

Update: I ran Mann-Whitney Test for Two Independent Samples since the normality of data was questionable and the results of the t-test were confirmed with non-parametric test too.

My Data are as follows:

G1     G2     G3    
37.770  30.94  20.040
22.945  32.55  16.790
53.230  51.83  41.740
21.960  20.99  16.325
25.920  19.11  20.710
40.290  34.74  23.380
28.960  28.21  21.490

Answer 1

In short: neither are appropriate and you shouldn't try to interpret them.

---

Measurements from the same experimental units are not independent.

If G1, G2 and G3 come from the same 7 fish, then you cannot use a test or model that assumes independent observations. This is no different for non-parametric statistics.

---

You can model this dependence using a linear mixed effects model.

By assuming that these fish come from a larger population of fish with random, normally distributed deviations from the 'average' fish w.r.t. the concentrations of these chemicals, we can estimate a random intercept for concentration.

In $\textsf{R}$, you can do this quite easily with the package lme4:

# A great package for mixed models
if(!require("lme4")){
  install.packages("lme4")
  library("lme4")
}

# A package for, among many other things, nice QQ-plots (including 95%-CI)
if(!require("car")){
  install.packages("car")
  library("car")
}

# Your data
M <- data.frame(concentration = c(37.770, 30.94, 20.040,
                                  22.945, 32.55, 16.790,
                                  53.230, 51.83, 41.740,
                                  21.960, 20.99, 16.325,
                                  25.920, 19.11, 20.710,
                                  40.290, 34.74, 23.380,
                                  28.960, 28.21, 21.490),
                bodypart      = factor(rep(1:3, times = 7)), 
                fish          = rep(1:7, each = 3))

# A Linear mixed effects model
LMM <- lmer(concentration ~ bodypart + (1 | fish), data = as.data.frame(M))

# Some basic diagnostics
plot(LMM)
qqPlot(residuals(LMM))

# A summary of the model
summary(LMM)

# Bootstrapped confidence intervals for difference from the reference group
confint(LMM, method = "boot")

# How to change the reference group:
M$bodypart <- relevel(M$bodypart, "2") # this would set G2 as reference group

---

Why does this matter?

Your standard deviations (and thus any claims of significance) can differ quite a lot depending on how your model your data. In some cases you will underestimate your standard errors and in others you will overestimate them.

Take for example your own data and try and run a normal linear regression and a mixed model, using the same fixed effect (bodypart):

LM <- lm(concentration ~ bodypart, M)
summary(LM)
summary(LMM)

See the large difference in standard error? That's what happens when you treat measurements of the same fish as though they were independent.

This answer is already getting quite long and I recommend you read up on random and mixed effect models, but I'll try to give you an intuitive interpretation of why this difference can be so large, simply by plotting your data:

By calculating a random offset for each fish, we get a much clearer picture of what the effect of each body part is on the concentration.

Of course you could argue that a linear model with a fixed effect for fish also sees that these are different fish, however: (1) Body parts are still very much nested in fish, so this wouldn't solve the dependence issue (2) Estimating 7 different offsets would be a huge waste of your small sample size, when instead you can estimate a single random distribution from which your fish' offsets come.

I hope to have convinced you of the need to study mixed models a bit before trying out study designs with dependency.