2
$\begingroup$

Question

What should my (multinomial logistic regression) model look like for analysis of the data described, and how can I interpret it and report it in an acceptable way for journals that typically expect a P value.

Background

I'm a grad student. I have taken the only stats course available in my department, so I am not very experienced with stats and R but I am trying to learn while also conducting analyses for my lab. We do linguistics studies, so we don't have many math-oriented people, and I'm the most informed on stats. I want to learn but I am struggling somewhat.

I have run and understood linear regressions at a basic level. The data I am trying to analyze now, though, are not continuous, and I think don't really work for linear regression. I'm trying to figure out how to implement a multinomial logistic regression model, although I'm not entirely sure that this is the correct approach.

The data

I am working with the results of an experiment. Each response has the fields defined below (in JSON because I'm a JavaScript nerd and md+ tables are not supported here, not because it has any concrete connection to this situation):

{
  participant: {
    description: "a label corresponding to a person who participated in the experiment",
    type: "categorical"
  },
  sentence: {
    description: "a label corresponding to an English sentence being read aloud by the participant.",
    type: "categorical"
  },
  attempt: {
    description: "denotes whether this response is a first or second attempt at reading the sentence aloud",
    type: "binary, ordinal (really, 1 or 2)"
  },
  score: {
    description: "how natural the attempt sounds in the view of a researcher scoring the recordings",
    type: "categorical (possibly ordinal): good | NP-good | VP-good | bad"
  }
}

My attempt at a model

What I want to know is, to what extent the possibility of an outcome (score = good | bad | VP-good | NP-good) is modified by the value of "reading (1 vs 2)" and I would like to "factor out" the effect of participant and sentence (that is, exclude variance that can be accounted for by those factors when considering reading).

In a mixed effects linear regression, I would want something like:

score ~ attempt + (1 | participant) + (1 | sentence)

i.e. the fixed effect is attempt, and the random effects are participant and sentence. However, I am not confident that the values of score are ordinal, and they are certainly not continuous. Trying to translate that to work with multinom as in this tutorial, I get:

score ~ attempt + participant + sentence

But I am not sure if this is really doing what I want it to. I'm also not sure how to understand the output. I can't figure out how to follow the final steps of the tutorial, because my predictors are not continuous and can't really be binned, but have many levels.

tl;dr

So: am I even on the right track here? Should I be using a different analysis? A different package? How can I get something reportable for a journal that typically wants a p-value?

Please be gentle, but all criticism and feedback is welcome :)

EDIT: Maybe I can refactor my model so that's it's not multinomial, where what I look at is predicting "reading (1v2)" using score as a factor?

$\endgroup$
  • $\begingroup$ Should have self study tag. $\endgroup$ – Michael Chernick Jan 31 '17 at 20:53
  • $\begingroup$ so the output variable you are trying to model can take on four possible levels? "good", "np good", "vp-good" and "bad"? you're ignoring participant, sentence, and attempt, correct? $\endgroup$ – Taylor Feb 1 '17 at 17:46
  • $\begingroup$ correct that my output variable has 4 possible levels. what I want to know, though, is to what extent the possibility of an outcome is modified by the value of "reading" -- and I would like to "factor out" the effect of participant and sentence $\endgroup$ – Tyler Peckenpaugh Feb 1 '17 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.