4
$\begingroup$

Sorry for the somewhat confusing title. I was wondering about the use of confidence intervals in the context of chemical and biochemical experiments. The experiments must be repeated and the data have to be independent, I know.

But - say you are doing an experiment where you mix two chemicals, let it react under some conditions, and measure for instance yield or purity of the product. If I now repeat the experiment, I will usually take the starting materials from the very same batch. Or maybe they are purified to contain 100 % pure starting material.

In this way, I can only see the calculated CI reflect uncertainties in concentrations, temperature, time etc. Will this be acceptable? If you had to take a blood sample of a mouse, of course you have to use several mice to get a CI. Here, all starting chemicals are identical.

Another case is if I take two different and pure chemicals and test e.g. toxicity on some cells. Will repetition of the experiment mean that I have to order several different batches of cells? Will each cell sample be like if I took a mean value of thousands of mice in each and every experiment?

BR Steffen

$\endgroup$
2
$\begingroup$

(Hi and welcome to cross validated)

You raise very good questions: the outcome of the experiments may depend on a large number of factors. Doing really independent replicates of the experiment is often not feasible from a practical point of view (ordering multiple times the same cell line, etc.).

However, here are some things you can do:

  • Report precisely the conditions of your experiment, and at which level your replicates are independent (and on which they are not), e.g. that your test cases were prepared from newly prepared stock solutions but from the same lot of the chemicals (= variance includes preparation error, but not differences between lots / manufacturers of educt).
  • From a pragmatical point of view, you could focus the replication experiments on being independent on the important factors. Unfortunately, there is not too much knowledge which factors are typically important. I'd suggest to speak to experienced experimenters where they see the critical points. The problem is that while experimental results on these questions would be needed, it is really hard to get funding to do this rather tedious = expensive in terms of wage, possibly also expensive in terms of materials purchase work which is unfortunately often not regarded as being of much importance - particularly if you are successful in showing that e.g. lot/manufacturer do not matter. Publication bias will be very much against you if you show that a replication study (= already low on the novelty scale) did not find differences (= on the first glance, nothing interesting was found, because patterns/differences are interesting, lack of them is not).

If I now repeat the experiment, I will usually take the starting materials from the very same batch.

This is IMHO a case for: speak with experienced experimenters in the field whether they's expect problems. If they say that, you may be able to convince your supervisor that you should replicate on the lot/manufacturer level.

Will this be acceptable?

Usually it will. Particularly if you make clear that you are aware of the limitations that implies for general conclusions.
Personally, I'm most concerned about people who have no sense for how "local" their results are. And the fact that your question shows you are aware and concerned about these makes me relax and trust your conclusions far more than I'd trust someone who claims to rescue the world on the basis of a single calibration with no whatsoever validation...

Will repetition of the experiment mean that I have to order several different batches of cells?

Yes. My experience (vibrational spectroscopy of biological samples/cells and what I have heard from colleagues who do microarray studies on such samples) unfortunately suggests that batch-to-batch variation for cells is indeed an important source of variance.

Will each cell sample be like if I took a mean value of thousands of mice in each and every experiment?

No, it will be as if you looked on several cells (of the same tissue) of the same mouse.

**Update ** wrt. to your comment: sure, central limit theorem applies when averaging a large number of cells (though be careful, proportions/count data take quite a number to approach a normal distribution.
However, averaging a large number of cells of the same batch will approach the mean of that batch only. If you have noticeable variation between batches, then averaging large numbers of cells of the same batch does not help in order to establish the batch-independent average (I just did an experiment where some kind of phenotypical variation between batches/biological replicates where in the same order of magnitude as the phenotypical variation between different tumor cell lines).

$\endgroup$
  • $\begingroup$ Thank you for your clear reply and flattering words! I have been twisting my brain the last few days. Regarding the last question, I think we talk past each other. I was thinking of the central limit theorem, but tried to hide my uncertainty. With respect to the cells, I see there must be an underlying probability function for each cell whether it dies at a given concentration or not (binomial?). If we could measure %survival from an apparatus based on a mean of all cells, I understand we will approach a normal distribution due to the CLT. But of course not if they are counted individually. $\endgroup$ – pseudoninja Jun 12 '16 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.