7
$\begingroup$

In our lab, we work with the retina which yields very little sample. Although we have a reasonable sample size of six animals in each group, I don't understand which test to be used to test for significance among groups. This is because we mix the sample (from all 6 animals) since it is too less in quantity. So, we mix the yield and then run a Western blot for it all together and then measure its intensity.

The main objective of our study translates to checking for differences in the Western blot result among different groups. Can anything be done for this, since even if I repeat the blot, it will take into consideration the variations in the western blot but not the variations in the population.

In short, we just have the mean blot intensity value for 6 animals (since we mixed the sample together - apparently that is how it is usually done with the retina. Edit - pooled is the word) but we lack any information on its standard deviation.

I don't know much of statistics but is there any test that can be used in this case? I know I can't use an ANOVA (which is generally used for such situations) because we don't have enough sample data (i.e. we lack the variance of our sample. we don't know how our sample is distributed) despite having adequate sample size.

Edit: The project we are working on has animals in different groups exposed to different experimental conditions. So I don't think predicting the variance from other studies is a good idea.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ I don't think there is a test that works without any variability in the model. $\endgroup$
    – SmallChess
    Commented Mar 23, 2017 at 9:04
  • 4
    $\begingroup$ Relevant publication: Prosser 2010, in particular excuse #viii (pooled replicates) and also #i (replicates too expensive). $\endgroup$
    – Roland
    Commented Mar 23, 2017 at 12:51
  • 2
    $\begingroup$ @Roland +1, fantastic publication which should be read by every supervisor and project applicant everywhere. "If sufficient funds are not available to do good science, then it is better to do no science than bad science." $\endgroup$
    – juod
    Commented Mar 23, 2017 at 13:57
  • 2
    $\begingroup$ Yes, there are several tests you can use. For an introduction into why this might be the case, please see stats.stackexchange.com/questions/1807, which deals with a simple (but illustrative) case of your situation. Moreover, repeating the composite measurement is relevant: that would enable you to perform a routine ANOVA to test whether the means in each group differ significantly--and isn't that exactly what you want to find out? $\endgroup$
    – whuber ♦
    Commented Mar 23, 2017 at 14:10

3 Answers 3

Reset to default
12
$\begingroup$

No, I do not believe there's any formal test that you can apply. (By the way, the usual term for such samples is pooled.)

You are right that repeating the blot will not tell you more about the biological variance in population. You could try to look up the variance obtained by others in similar, but non-pooled experiments (if there is any), and use it to make a very rough and informal test. But I would suggest you to stick to non-statistical comparisons, because to achieve significance with $n=6$ the differences would need to be obvious anyway.

EDIT - would like to clarify my reasoning, in response to good points by @whuber below: I agree that any comparison, or any logical reasoning at all, should follow basic rules of statistics and probability. Probably a better way to express what I (and @Joshua?) meant is that at small $n$, conclusions are very dependent on the priors. In the given situation, using my experience, I could make some guess of the variance in protein expression between healthy animals, but if $n=1$, I have only my guess to plug in a test. With $n=3$, I might still want to check whether the estimated variance corresponds to my expectations, and with $n=100$ I throw the guess out the window, because the estimate by now is likely more reliable than my expertise.

While the Bayesian framework can incorporate all that, converting expertise to actual numeric values is very complicated, and we end up with a conclusion "groups look kinda different". That is what I meant by a non-statistical comparison. It is certainly crude, but I believe that the discussion section of any study is just such eyeballed conclusions combining data with prior knowledge.

Improve this answer
$\endgroup$
6
  • $\begingroup$ All comparisons are statistical: there is no such thing as a "non-statistical" comparison. There is such a thing as ignoring the statistical facts (willfully or not). If a comparison is "obvious" then there must be valid statistical support for it. Otherwise, your answer amounts to advocating a selective use of statistical procedures. That leads to arbitrariness and subjectivity. $\endgroup$
    – whuber ♦
    Commented Mar 23, 2017 at 14:14
  • $\begingroup$ @whuber: Not so. The sample size is so small that intuition of looking at each individual case will yield better quality data than the statistics tests. $\endgroup$
    – Joshua
    Commented Mar 23, 2017 at 15:33
  • $\begingroup$ @Joshua That's a completely anti-statistical statement: it amounts to denying the validity of the entire discipline of statistics. It raises what appear to be insurmountable questions, such as (1) How do you decide when a sample is so small that you should use intuition? (2) Whose intuition do you use? (3) How do you resolve situations where two people's intuitions give conflicting answers? (4) How do you quantify an intuitive answer? (5) How do you determine how good an intuitive answer might be? $\endgroup$
    – whuber ♦
    Commented Mar 23, 2017 at 15:48
  • $\begingroup$ @whuber: I've seen too many cases where n was set sufficiently high to get really good confidence where a few hours research in the library would have revealed the results were terrible. The math wasn't wrong. The assumption of random selection was. If n is small enough to practically check for outside variables in each case one should do so, and doing so will always yield better confidence than not doing so. $\endgroup$
    – Joshua
    Commented Mar 23, 2017 at 17:50
  • 1
    $\begingroup$ +1 for the clarifications in this answer. Certain kinds of assumptions (such as distributional ones) become more important to the analysis as the sample sizes go down. I also agree we have to be realistic about the limitations of statistical procedures when there's little information. I just wanted to make it very clear that this never gives one license to abandon statistical thinking and statistical approaches altogether in favor of "intuition," no matter what the sample size or assumptions might be. $\endgroup$
    – whuber ♦
    Commented Mar 24, 2017 at 15:37
4
$\begingroup$

There is probably not much to be done.

If there is some way to estimate the within-group variance of unpooled western blot results from previous data, you could use this to generate an estimate of the standard deviation for the pooled western blot and use that to run a test. But it does not sound like this is the case.

As an interesting aside I offer that you could generate a confidence interval for the difference in means using the methodology explained here : What can we say about population mean from a sample size of 1?

However, since this CI will always include zero, this will not be of much help in terms of significance testing.

Improve this answer
$\endgroup$
2
  • 3
    $\begingroup$ There are single-sample CI's that do not necessarily include 0. See arxiv.org/abs/bayes-an/9504001. $\endgroup$
    – whuber ♦
    Commented Mar 23, 2017 at 14:12
  • $\begingroup$ I will need to take a look at that when I have a moment of time. It's always surprising what is possible. Thanks! $\endgroup$
    – Erik
    Commented Mar 23, 2017 at 14:38
-1
$\begingroup$

I would look into nonparametric statistic to find a test that would work for your situation. These tests don't have as many parameters to match to make the test work and are designed to work for smaller sample sizes.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Welcome to Cross Validated! Please take a moment to view our tour. Our preference is for longer answers that are self contained with citations to substantiate the claims. Please consider expanding your answer. $\endgroup$
    – Tavrock
    Commented Mar 23, 2017 at 18:52
  • 1
    $\begingroup$ A non-parametric test usually uses less assumptions. At least for small n that means less power than a parametric test. And if n gets too small (with n = 1 being the extreme discussed here) they don't work at all. $\endgroup$
    – Roland
    Commented Mar 24, 2017 at 7:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.