1
$\begingroup$

I'm assessing effects of age, gender and speech style on the realization of linguistic forms. All the dependent and independent variables are binary categorical. My dataset:

enter image description here

enter image description here

When I run logistic regression, I get the output with highly significant effects of 2 variables:

enter image description here

enter image description here

If I add 3-way interactions to the model, all p values jump to around 1:

enter image description here

enter image description here

I'm stumped by this transformation and I wonder what could affect the output so much. Can the model with 3-way interactions be unreliable? I read that a big dataset might be needed to assess interactions, otherwise, things can get messy. The sample of speakers I'm working with is 16. The number of data observations is 1164.

$\endgroup$
  • 1
    $\begingroup$ Could you please show the specific function calls that you invoked for the logistic regression, and a sample of your data? That might help in providing an answer. Note that the standard errors in the interaction model are extremely high. $\endgroup$ – EdM Jul 3 at 15:59
  • 1
    $\begingroup$ Please provide some details of your model--I cannot reproduce these results with the data shown in your plot. The actual counts are needed. What is the link function? What do the 0/1 codes mean for each factor? Do you mean something special by "linguistic regression"? How are you handling the correlations likely introduced by having multiple observations per speaker? $\endgroup$ – whuber Jul 3 at 15:59
  • $\begingroup$ If you have 16 subjects with 1164 total observations you must have have repeated measures. How are you controlling for that ? $\endgroup$ – Robert Long Jul 3 at 17:12
  • $\begingroup$ @EdM I added the functions, could you please have a look a them $\endgroup$ – Polina Jul 3 at 17:30
  • 1
    $\begingroup$ You haven't given enough information about your study design to be sure of the details, but in general yes random intercepts are a good way to handle repeated measures. $\endgroup$ – Robert Long Jul 3 at 18:26
1
$\begingroup$

Although adding interaction terms to a regression model can change coefficient estimates and p-values from those estimated without interactions for several reasons, the spectacular increase in the standard errors suggests that your data suffer from the phenomenon of perfect separation that can occur in logistic regression. That is, your combination of predictors completely predicts the binary outcome. Then the maximum-likelihood method for fitting the model runs into problems and standard errors of coefficient estimates can be enormous.

It might seem that perfect prediction is a modeling success, but typically results from such a model don't work well on a new data sample. This discussion goes into more detail and suggests ways to deal with perfect separation.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you so much! I didn't have a clue about such a phenomenon. $\endgroup$ – Polina Jul 3 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.