# Significance calculation between two sets of before and after measurements. Does one decrease more than the other?

Some background:

I am studying the immune system. This is the defense the body has to protect itself from pathogens like bacteria and viruses. The main cells involved are commonly known as white blood cells.

What we measure: We generally measure the strength of people’s immune response. We do this by:

1. taking a vial of blood from an individual
2. introducing a pathogen (virus, bacteria, fungus) to the blood and,
3. measuring the amount of certain proteins produced by the white blood cells. More protein is considered a stronger response.

Our first hypothesis:

We suspect a specific environmental situation might temporarily decrease the response strength of the immune system in response to certain stimuli. We want to test this by first taking blood from people at baseline, then putting them in the specific environmental situation for a few minutes, and after taking them out of the situation we take some blood again. This way we can compare before and after. We will stimulate the blood with different pathogens and compare the protein production before and after. To statistically compare these I can use a paired t-test or Wilcoxon signed-rank test.

Our second hypothesis (the one my post on this forum concerns):

We also suspect specific pathogens to be effected more by the specific environmental situation than others. Say for instance we expect pathogen A to be more effected than pathogen B. My question is: what statistical tests can I use to determine if the effect is larger in A than in B? There is one more thing to take into account, namely that the baseline levels of immune response to pathogen A and B will not be the same. E.g. pathogen A might induce 3 times the amount of protein compared to B at baseline. This means we will have to work with ratios or percentages.

My thoughts:

I was thinking I might calculate the percentage decreases or ratios/fold-changes per person for both pathogen A and B and compare these with a (non)parametric t-test. However, I am not sure if this is correct, and if I should use percentages or ratios/fold-changes. Also, we know age and gender effect responses, so should we correct for these? This situation might compare to some drug trails, where they compare the effects of drug A vs drug B (e.g. cholesterol lowering), but here the baselines will be more comparable.

I hope I was clear, if not, please feel free to ask,

Thanks in advance, Have a wonderful day,

Rob

PS. Some examples of drug/treatment studies: In this study http://jamanetwork.com/journals/jama/fullarticle/198311#JOC32467T3 they study the effect of lipid lowering drugs. As a test they use an analysis of covariance model applied to rank transformed data (see Table 3). This other somewhat similar study http://care.diabetesjournals.org/content/28/7/1547.full treatments were compared using least-square means.

There probably is no definite answer to this questions, so this post is just my (kind of lengthy) train of thought, playing through several scenarios.
First off, regarding your last question

Also, we know age and gender affect responses, so should we correct for these?

This depends on the goal of your study, as well as the number of your probands and can't be answered by us with certainty. You also don't specify, whether each proband's response is tested for both pathogens or just one.
If each proband is tested with only one pathogen, you would have to make sure that the control groups are comparable - else, the outcome of the study can't be clearly attributed to the different environment.
In this case, some (obvious) possible solutions are:

• Split the test subjects into groups, according to age and gender. (not possible, if you have only a small amount of probands)
• Only work with comparable probands, e.g. male, 30-40 years old. (probably the most sensible approach, if you don't have funding or enough people for a large-scale study).
• Make sure that the groups are split in a comparative way, i.e. about the same number of men and women, spread equally across the age groups.

The above will probably not be necessary, if you can test the pathogen response behavior to $A$ and $B$ for each individual test subject (which would be the preferred option).

Now that the groups are set, you could apply the following procedure:

1. Measure the immune responses $R_{A/B}$ to pathogen $A$ and $B$ under standard conditions. Calculate the mean responses $MR_{A/B}$ to each pathogen and check, if there is a significant difference between $R_A$ and $R_B$ (by applying a t-test, for example). This is the baseline reaction to the pathogen.
2. Next, you check the responses $\tilde{R}_{A/B}$, after they have been submitted to the environmental stimuli. This time, check for a difference between $R_A$ vs. $\tilde R_A$ and $R_B$ vs. $\tilde R_B$. (t-test, etc.)
3. Let's assume for the last step that the previous test showed that the different environment indeed significantly affects the immune response (otherwise the study would be pointless).

what statistical tests can I use to determine if the effect is larger in A than in B?

If the test in 1) returned that the baseline reaction to $A$ and $B$ is equal, we could simply test if $\tilde R_A$ is significantly different from $\tilde R_B$ (t-test etc.). If this is the case, we compare the difference in mean immune response under the environmental influence: $\widetilde{MR}_A-\widetilde{MR}_B$ and follow that the environment has a bigger influence on pathogen $A$ or $B$.

This is option is not very likely, though, as the immune response $R_A$ is probably different from $R_B$.
Hence, we have to compare them some other way. To do this, we could for example either rescale them in absolute terms, i.e. consider $\tilde R_A-MR_A+MR_B=:\tilde R_A(B)$, or in relative terms, i.e. $\tilde R_A\cdot\frac {MR_B}{MR_A}=:\tilde R_A(B)'$. Then, we could again apply a test, to check for a difference between $\tilde R_A(B)$ or $\tilde R_A(B)'$ and $\tilde R_B$.
Which one is more appropriate is up to you to decide and depends, on whether you think that a larger response translates to a larger environmental influence (then use $\tilde R_A(B)'$), or not (use $\tilde R_A(B)$).
If the previous test suggests that there is a difference, you can calculate the mean of the rescaled response to pathogen $A$ and compare it to $\widetilde{MR}_B$, concluding that the study results suggest a bigger environmental effect for $A$ or $B$.

• Thank you @Eldioo. First let me clarify, you asked "You also don't specify, whether each proband's response is tested for both pathogens or just one." . We do measure both pathogens in each individual. 3 questions: (1) You mention: "Which one is more appropriate is up to you to decide and depends, on whether you think that a larger response translates to a larger environmental influence ... or not ...". I would say that it is the percentage decrease that matters, so that corresponds to the "relative terms" option, right? (2) Would you use "standard" (rank-based) t-tests? Commented May 8, 2017 at 9:02
• (continued) (3) In your opinion, would the tests used in the two studies I mentioned, especially the ANCOVA on rank transformed data be appropriate also? Commented May 8, 2017 at 9:06
• Regarding (1): Yes (I also think that the relative term is more sensible). (2): A t-test requires certain assumptions, such as independent and roughly normal distributed samples. As both pathogen reactions are tested for each individual, a standard t-test might be inappropriate, as this likely results in correlated values for $R_A$ and $R_B$. Thus, using the Wilcoxon signed-rank test probably is the preferrable choice. Commented May 8, 2017 at 10:20
• (3) I only skimmed through the papers and they seem to both compare two independent samples, each patient only getting one treatment. One of the studies uses a t-test (not appropriate here) and the other uses a wilcoxon signed-rank test, as suggested. Regarding ANCOVA: I guess using it to filter for the effect of the pathogens makes sense. Commented May 8, 2017 at 10:25
• Thanks! I think I'll give the Wilcoxon signed-rank test a go. Commented May 12, 2017 at 14:40