What to call this dependent variable calculated as the relative difference between two numbers?

Question

I ran an experiment where I expected that reaction times in two different conditions would differ, and I estimated that one condition would always be associated with longer reaction times.

After collecting the data, I realised that there did appear to be systematic differences between the two conditions, but that individual differences (between participants) seemed to dictate the direction of the difference (i.e., some people responded systematically slower in one condition, and other people vice versa). I mostly just care that differentiation is happening, not so much whether reaction times are shorter or longer in one condition or the other.

What I would like to do is calculate the relative distinction (in reaction times) between conditions. I don't want to use the absolute or unsigned difference in milliseconds, given that different participants will have different reaction time speeds in general.

In my manuscript, I ended up dividing the mean reaction time of condition A by the mean reaction time of condition B, and then used the unsigned difference between that ratio and 1 as the new dependent variable:

the absolute value of 1 - [condition A/condition B]

I received back reviewer comments and they didn't feel I named or communicated this term very well. I said "ratio difference" but I can see why that would be confusing. So I need a standard term, or a more concise and clever definition than I can come up with. Is there a commonly accepted statistical term I have missed?

The other criticism was a lack of significance testing. How can I establish, for each participant, that this ratio is significant, before adopting it as a dependent variable? For each condition (A or B), for each participant, for each block, I have at minimum 30 reaction times. So perhaps I could test, within each block and participant, between the two conditions, as is suggested by this answer. In that case, should I use the test statistic as a DV, or simply report it as validation for using the ratio difference? And I presumably would then correct each participant test's p-value for multiple comparisons.

Edited to add that I have been looking at questions already posted here concerning the use of ratios in regression, but it seems that this is an issue where the ratio is constituted by an interaction, especially if the ratio is meant to be interpreted as a predictor. What I am chiefly concerned with is how unequal the two conditions are, if that makes sense--not necessarily the relative contribution of one factor over the other.

Edited again to add: for anyone else reading this question with a similar issue, these previous questions do a much better job articulating what my original research goal was:

Extracting slopes for cases from a mixed effects model (lme4)

Test whether random slopes are significantly different from 0 for individual subjects

Answer 1

First, your formula can be written in several different ways:

$$ \left| \frac{A}{B}-1\right|=\left|\frac{A-B}{B} \right|=\frac{\left|A-B \right|}{|B|}$$

with the last being the form for one definition of (absolute) relative difference.

Second, that form places a particular importance on Condition B as the reference. If that's the case your formula might make some sense. But if Condition A is really the reference you might want to replace the denominator by |A|. If neither is a reference, you might be better off using the maximum of A and B or their average for a measure of relative difference.

Third, your use of this measure on the averages as the outcome for each participant might be getting in the way of a statistical approach to test your hypothesis. In general, the closer you get to the original observations, the better. The multiple trials for each participant suggests that you might be better off with going back to individual data rows for each trial.

Each data row then represents a single trial with the reaction time, the Condition, the ID of the participant, and any other covariate values you might have. If you think that these are relative differences rather than absolute differences you might want to work with a log scale for reaction times, or use a generalized linear model with a log link between the predictors and the outcome.

The two Conditions are a fixed effect, with one chosen as the reference. The random intercepts take into account participant differences in the reference-condition reaction times, and the random slopes would represent the further participant differences between the two conditions. The magnitude of the variance among those random slopes seems to be what you are interested in.* For an introduction to testing significance of random effects, you could start with this page.

Finally, I'm a bit worried by the way you have approached this. You started out with an hypothesis about a particular directionality of difference between the two Conditions, and only came up with this outcome measure after you found that hypothesis to be unsupported. That's not generally good practice, as hypothesis testing fundamentally assumes hypotheses that were formulated without looking at the data.

---

*Generally one would expect a distribution of differences between 2 conditions among individuals just at random. I expect that you have some variability of a particularly large magnitude in mind, or perhaps a difference that in turn is associated with some covariate for each individual.