Statistical tests for low sample numbers in both indendent groups (n=3 and n=1)

Question

I've been asked to analyze some data for an experiment that's already been done. 15 rats were raised in 5 groups of 3

1. No treatment
2. Saline treatment (control drug is delivered via saline)
3. Drug treatment
4. Saline shock (high dose)
5. Drug shock (high dose)

Due to budget constraints, groups 2~5 were physically pooled (i.e. the three samples in each group were mixed and analyzed as a single sample) so as to fit onto a single iTRAQ 8-plex, whereas group 1 was not pooled so I have 8 channels

1. No treatment
2. No treatment
3. No treatment
4. Saline treatment (control drug is delivered via saline)
5. Drug treatment
6. Saline shock (high dose)
7. Drug shock (high dose)

For those of you who don't know what iTRAQ is, it's a mass spectrometry protein identification and quantification method, which generates a list identified proteins and an 'absolute' quantification value. I say absolute, but it is meaningless in an absolute sense, and only useful when compared to another quantification value in the same experimental run. So for a given protein X I can calculate the ratio of X in channel 5 to the ratio of X in channel 1, or even the ratio of X in channel 5 to X in group 1 (average of channels 1,2 and 3)

$$X_5/(\sum_{n=1}^{3}X_n)/3$$

But what I want to know is if there are any statistical methods I can use to determine whether a given protein ($X_5$) is significantly different (in quantity) from ($X_{1,2,3}$).

Having only three samples with no treatment obviously means establishing any variance is probably meaningless, so I was thinking that some non-parametric rank based test might be the way to go? What is the best test for this setup?

Answer 1

I'm sorry to say that I can't help but read "due to budget constraints, the experiment was rendered useless." I'll try to explain why.

A nonparametric rank-based test would be to assign ranks to $X_1,X_2,X_3$ and $X_5$ and to see if the rank of $X_5$ is higher than what you would expect if there was no different between 1-3 and 5. Unfortunately, with only four observations in total, you would get a p-value of 0.25 in the most extreme case, i.e. when $X_5$ is greater than each of $X_1,X_2,X_3$. That's the lowest possible p-value with this experimental design. In other words, this approach has low power - in fact, so low that it never can detect differences.

You could use a two-sample $z$-test if you think that you can assume normality and make reasonable assumptions about the variance of the measurements. That would make use of the actual measurements and not just their ordering and could therefore give lower p-values. On the other hand, it will be sensitive to your assumptions about the variances and may not work well if your data isn't normal. Obviously the variance is impossible to estimate for $X_5$ (since you only have one measurement) and you can't assess normality when you only have a few observations. Perhaps you have some prior knowledge in those directions (variance and normality) that you could use, e.g. from previous experiments, knowledge about these proteins and the quantification method? Otherwise I think that the researchers will have to accept that they've just been given an expensive lesson: you should always plan the statistical analysis before carrying out your experiment.