1
$\begingroup$

My dependent variable is reading time. My two predictors are categorical.

I conducted a lmer model with the default family and the performance package indicated that the response distribution fits better with an inverse gamma family. enter image description here

So I conducted a glmer model with the gamma inverse family: enter image description here

And I conducted a glmer model with the gamma identity family: enter image description here

The formula was always the same, but the in the plot of the gamma inverse model seems almost like the symmetrical mirror of the x and y axis of the gamma identity. Is this a usual thing?

UPDATE:

I did not transform my data before running the models, but the RT are the sum of three reaction times:

The origin of the data is the measurement of reaction time for button presses in seconds. Which button press would reveal to the participant a new word of the sentence. I am following previous experiments, and the dependent variable is obtained as the sum of the reaction time of the last three words of the sentences (these are the words after we introduced the manipulated word). The purpose is to see in which conditions participants are slower and in which conditions they are quicker to read after the manipulation. (I personally think a gam model with the reading times predicted by word position and predictors would be more interesting here to see if the line tendency is to increase or decrease reading times in each conditions, but I am only a student.)

Model1:
minv <- glmer(RT ~ 1 + pred1 + pred2 + pred1:pred2 + (1 | id) + (1 | stimulus_id) + (1 | order), data=df, family= Gamma(link="inverse"), control = glmerControl(optimizer = "bobyqa", calc.derivs = TRUE))

cat_plot(minv, pred = pred1, modx = pred2, geom = "line", line.thickness = 2, interval = FALSE)

Model2:
mid <- glmer(RT ~ 1 + pred1 + pred2 + pred1:pred2 + (1 | id) + (1 | stimulus_id) + (1 | order), data=df, family= Gamma(link="identity"), control = glmerControl(optimizer = "bobyqa", calc.derivs = TRUE))

cat_plot(mid, pred = pred1, modx = pred2, geom = "line", line.thickness = 2, interval = FALSE)

Histogram of the reading time in each condition: enter image description here

$\endgroup$
11
  • 2
    $\begingroup$ In a comment to your previous question, @Glen_b explained that the model(s) you are fitting are for the conditional response. In other words, don't plot a histogram (and why does the histogram have only 2 bins?) for all responses lumped together. It's more appropriate to make one plot for each of the four groups determined by your two binary predictors. $\endgroup$
    – dipetkov
    Mar 23, 2023 at 10:36
  • 1
    $\begingroup$ As for the inverse relationship, it's to be expected as even though the formula $X\beta$ (on the right side) is the same, the link function is different. With identity link $\operatorname{E}(Y) = X\beta$ while with the inverse link $1 / \operatorname{E}(Y) = X\beta$. In your case $X\beta$ is very simple because you have 4 groups determined by your 2 binary predictors. $\endgroup$
    – dipetkov
    Mar 23, 2023 at 10:50
  • 1
    $\begingroup$ I think I understand the question better now. In your plots the y-axis is labeled "RT". But do you plot the response on the original scale (the reading time scale) or the linear scale (the $X\beta$ scale)? $\endgroup$
    – dipetkov
    Mar 23, 2023 at 11:05
  • 2
    $\begingroup$ You use the inverse link, so yes, you are doing a kind of transformation. I would suggest to follow @SextusEmpiricus advice and show your code. $\endgroup$
    – dipetkov
    Mar 23, 2023 at 12:30
  • 1
    $\begingroup$ What is this graph 'distribution of response'? Are those the observed reading times? Is that reading time like a variable with only values 1 or 2? $\endgroup$ Mar 23, 2023 at 13:03

1 Answer 1

0
$\begingroup$

Is this a usual thing?

If you are plotting different models then you are gonna get different results. And yes, this might even completely reverse the effects.


That's a danger of the practice that you are using here. You seem to be plotting many different models until you find something that looks ok. Ideally you have some previous ideas about what model should be neccesary, with some technical/theoretical motivations, rather than picking out whatever model seems to give the best outcome for your case.

$\endgroup$
5
  • $\begingroup$ Reconsidering my answer, it is actually weird that the value of RT as a whole is larger/smaller. The lowest point in the one graph is larger than the highest point in the other graph. So probably your RT plots are incorrect and one of the graphs shows 1/RT instead of RT. But still you do get that the methods differ. If you convert the axis of one of those graphs into 1/RT then slopes of the lines will change. $\endgroup$ Mar 23, 2023 at 11:04
  • $\begingroup$ If you give more details about the code that you use to produce these images then this might be verified. $\endgroup$ Mar 23, 2023 at 11:05
  • $\begingroup$ I agree that pre-registration of the methods and analysis are critical. I indeed have some previous ideas about the model: previous researchers that used the same experimental task used a simple Gaussian distribution, and other researchers in my field advised the use of an inverted Gaussian with the identity link. However, none of those families fit my data well. I am looking for a model that fits my data, so I can trust the results. $\endgroup$
    – Olivia
    Mar 23, 2023 at 12:30
  • $\begingroup$ @Olivia "However, none of those families fit my data well. I am looking for a model that fits my data, so I can trust the results." sure, but it is a practice that needs to be done carefully. It would be cherry picking if you would continue an analysis on the same data (I am not sure whether this are your intentions, but it needs to be mentioned for anybody else that comes across this question and might pick up that this is a good way of analysis). $\endgroup$ Mar 23, 2023 at 12:57
  • 1
    $\begingroup$ Another thing, instead of working with weird distributions,l and link functions, why not model directly reading speed instead of reading time? What is the source of error in your experiment, is it your stopwatch (which would mean that using RT as variable makes sense) or is it the sampling of people (which might possibly make speed = 1/RT a better variable). Also, could you plot the reading times as a histogram for the four different cases? $\endgroup$ Mar 23, 2023 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.