My research question is: is the rate of forgetting (slope) influenced by the degree of encoding? So I have three encoding degrees and three time intervals in which I test people using different items.

Hence, what I need to know is if there is an interaction between the degree of encoding and time interval.

My items are 48 words, and I test 16 different ones at each time interval.

First, I fitted a model like this:

model1 <- brm(
  correct | trials(total) ~ 1 + time * encoding 
  + ( 1 + encoding | ID ),
  data = data,
  family = binomial("logit"),
  file = "model1"

But then, I realised I need to consider that there will be heterogeneity between the items because some are easier to remember than others. So instead of averaging across items (that will be very much like doing an ANOVA?) I tried to use the 0s and 1s (correct and incorrect responses) to fit another model:

model2 <- brm(
  correct ~ 1 + time * encoding 
  + ( 1 + time | ID ) 
  + (1 + time + encoding | item),
  data = databinom,
  family = bernoulli(),
  file = "model2"

I was happy with this, but reading a vignette in brms, I found a comment about a model like this that says: this model completely ignores the guessing probability and will thus likely come to biased estimates and predictions.

However, the words I use don't have 50% chance of being right, because it's not a recognition test. They literally have to write the word down to consider it correct.

All of the above raises the question, do I want to use the binomial family and just forget about the fact that the items are all different? Or should I use the Bernoulli one, which then will tell me the probabilities of getting each item right or wrong, but nothing about the rate of forgetting, which is the number out of 16 that the subjects get correct at the three time intervals.

  • $\begingroup$ Can you explain the experimental paradigm a bit more? Is this free recall? $\endgroup$ – jerlich Nov 17 '19 at 2:27

In this vignette the author of brms explains how to deal with guessing.

If the guess rate is 50% then

fit <- brm(
  bf(answer ~ 0.5 + 0.5 * inv_logit(time*encoding),
     time ~ (1| ID ) + (1|item), 
     encoding ~ item,
     nl = TRUE),
  data = data, family = bernoulli("identity"), 
  prior = c(prior(normal(0, 5), nlpar = "time"),
            prior(normal(0, 5), nlpar = "encoding"))

I'm not totally sure this is the way you want to specify the random effects or the priors, but the general formulation of a "mixed model" guessing mixed with should help you out.

| cite | improve this answer | |
  • $\begingroup$ I do not know the rate of guessing. In my study, people are not recognising responses. They have to actually remember the word, just like that. Different words have different difficulties, but I don't have the slightest idea of how likely each one is. There's no way anyone on this planet can tell how likely it is to remember the word "block" and how likely it is to remember the word "president". With the guess rate been unknown, can I still use the bernoulli family without having to state how likely each word is to be remembered? $\endgroup$ – Lili Nov 16 '19 at 21:53
  • $\begingroup$ sorry, I misread your question! I thought you said "the words I use have a 50% chance of being right" $\endgroup$ – jerlich Nov 17 '19 at 2:25
  • $\begingroup$ I used free recall and then cued recall. So first they freely try to remember, and after that, I suggest semantic categories in which the words fit. For example I say "a domestic animal" and they remember the word "cow" :) $\endgroup$ – Lili Nov 22 '19 at 14:18
  • $\begingroup$ It's a lot of work but you could generate a prior for the cued recall. Just give people who have never studied the initial word list the cue, and then you have the background rate of saying each word. E.g. if the word to remember is "hamster" and 5% of people say hamster when asked to name a domestic animal, this seems a reasonable guess rate. For free recall, this is a bit harder, but you could use some known word frequency database. $\endgroup$ – jerlich Nov 25 '19 at 8:31
  • $\begingroup$ Well, it is a lot of work and I have to do this for many experiments with different material. It would be like doing another complete experiment for each one of these just to find priors. Thank you anyway. $\endgroup$ – Lili Nov 26 '19 at 17:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.