Normalizing reading times using analysis of variance

Question

I'm trying to compare gaze durations measured using an eye tracker for each word. Since longer words will naturally lead to longer gaze durations, simply comparing the values is inaccurate.

gaze duration    word length
100ms            6
250ms            8
150ms            7

A simple way to normalize the values is by dividing the duration by the number of characters. However, this transformation assumes that reading time is normally a linear increasing function of the number of character, with a value of zero when the number of characters is zero which is not quite right.

gaze duration    word length    Adjusted gaze duration (duration / length)
100ms            6              16.67 ms/character
250ms            8              31.25 ms/character
150ms            7              21.4 ms/character

The best way to normalize the gaze durations was described in this paper:

A more nearly adequate position assumes that reading time is normally a linear function of number of characters, with a zero intercept. A linear regression analysis could be used to estimate the slope and zero intercept of such a function, and thus to estimate the expected reading times for regions of varying lengths. Deviations from these expected times would indicate the existence of factors that speeded or slowed the reading of any given segment.

Such an analysis was performed by computing the linear regression equation expressing reading time for each segment in each experimental passage as a function of the number of characters in it for each subject. The correlation averaged over all subjects was .38. The regression equation was used to obtain expected reading time on the basis of number of characters alone for each segment. The expected reading times were then subtracted from the obtained reading times and the resulting difference scores were submitted to an analysis of variance.

linear fit {6,100},{8,250},{7,150} = 75x - 358.333

gaze duration    word length    expected gaze duration    difference
100ms            6              75(6) - 358.333 = 91.66   8.34
250ms            8              75(8) - 358.333 = 241.67  8.33
150ms            7              75(7) - 358.333 = 166.67  -16.67

My question How does submitting the difference scores to an analysis of variance finally allow us to get the transformed values? Is merely using the difference scores as the normalized scores considered an adequate normalization?

Note The best answer should provide an example of the calculation necessary to get the final corrected values, you may use the sample values provided in the question.

Answer 1

I don't believe that normalization is what you want here. What you want to do is let your model account for the variance associated with the effect of word length on gaze duration.

I presume all of this occurs in the context of interest in the effect of other predictor variables on gaze duration, and you simply find the variance associated with word length to be nuisance variance. In this case, I suggest you look into using mixed effects models with crossed random effects (participants and word). This so far doesn't have anything to do with word-length effects, but as discussed here, you should be able to soak up a lot of nuisance variance associated with both participants and words by this approach.

To account for the possibly non-linear effect of word length, I suggest you use generalized additive mixed effects modelling, which should automatically figure out what degree of non-linearity is supported by the data. If you use the dev version of my R package ez, you can specify word length as a covariate via:

my_mix = ezMixed(
    data = my_data
    , dv = .(gaze_duration)
    , random = .(participant,word)
    , fixed = .(predictor1,predictor2)
    , covariates = .(word_length)
)
print(my_mix$summary)

However, note that using word length as a covariate assumes that:

1. word length isn't correlated with any of your other predictors of interest
2. the effect of word length doesn't interact with any of your other predictors of interest.

Assumption #1 might be assessed by:

my_mix = ezMixed(
    data = my_data
    , dv = .(word_length)
    , family = poisson #because word length is a count
    , random = .(participant,word)
    , fixed = .(predictor1,predictor2)
)
print(my_mix$summary)

and assumption #2 might be assessed by:

my_mix = ezMixed(
    data = my_data
    , dv = .(gaze_duration)
    , random = .(participant,word)
    , fixed = .(word_length,predictor1,predictor2)
)
print(my_mix$summary)

Addendum

You followed up that you wanted to account for the word length effect then use the subsequent word-length-eliminated data as features in a machine learning context. To do this, I'd run a gam and use the residuals as your features data:

library(mgcv)
fit = gam(
    data = my_data
    , formula = gaze_duration ~ s(participant,bs='re') + s(word,bs='re') + s(word_length,k=max(my_data$word_length),bs='ts')
)
my_data$resid = residuals(fit)