Which statistical test to choose - protein analysis

First I will describe my problem (1), then solutions I found (2), and then pose the question (3).

1) I am a student making a simple program for statistical analysis of data for a biological laboratory. The situation is as follows: They have a protein and measure some property three times (to eliminate measurement error). Then they mix the same batch of protein with some chemical and measure the same property again twice. In the end they want to know if the measurements are different enough. (I know the number of measurements is low, but it is the maximum they can do).

2) If we assume that the samples are independent I would choose Welch's t-test, because we cannot assume the same variance. However they say that, due to the fact that we measure the same batch of protein three times, and then two times with some chemical, the before and after measurements are related samples. But for related samples I only found paired t-tests, which cannot be used here, because there are no explicit pairs. The only thing that comes to my mind is to pair each measurement from each set.

3) How does one measure statistically the difference of a property and test whether that difference is statistically significant given this design? We have 3 measurements of batch of protein before applying a chemical and 2 measurements after applying a chemical. Measurements are repeated to eliminate error in measurement device.

I think the word "repeated" is confusing you. You have replicate measurements under two different conditions. Compare with an unpaired t-test. Whether or not to use the Welch t test is a matter of opinion.

You would only use a paired t test, as you point out, if each value in one condition was paired with one in the other (or the same individual measured befroe and after an intervention). This is not your experimental design. You have replication, not repeated measures.

The design imbalance is not a cause for concern. The precision of the mean-estimate is enhanced by drawing more samples. If the "post" assay is more precise then such a design may in fact be optimal.

You can perform a paired t-test in the aggregate by averaging the 3 replicates from the "pre" sample and the 2 from the "post" sample, then taking their respective difference. Alternately, you may choose a mixed model with a random intercept for each protein sample, 5 observations per cluster and an indicator variable for administration of the chemical. Another approach is to use a GEE with an exchangeable correlation structure for replicate assays of each protein sample. Since you use a Welch's paired t-test, the variance is calculated from this sample.

The standard error of the paired mean difference may not correspond to the standard deviation of either assay, but rather a complicated combination of them. What's important is that the confidence intervals for the mean difference are correct and the inference you draw from the sample makes optimal use of the design.

The Welch's T-test does not require homogeneity, but it does require independent, identically distributed samples within either group. A reason this may be violated is due to a mean variance relationship. I find this is almost always the case in biological samples. Be sure to inspect the distribution of residuals vs. cluster-specific mean differences. Higher concentrations of proteins tend to be more variable simply because the measurements arise from a counting process. Performing a log transform of the outcome prior to calculating means or differences can approximately stabilize the variance.

• ) Your suggestions assume the entire experiment (with n=3 in one condition and n=2 in the other) was repeated a few times. The OP suggests that this didn't happen and isn't going to happen. – Harvey Motulsky Dec 26 '17 at 18:16
• @HarveyMotulsky I had assumed this 3:2 design was replicated in separate proteins as a means of validating the assay. This precludes any reason to mention pairing. – AdamO Dec 26 '17 at 18:20