2
$\begingroup$

I have data about around 100,000 protests nested within 40 countries and I want to analyze when the claims of a protest are directed at the state, based on action and country level characteristics, using glmer. So, I would like to obtain p-values of both the fixed and random effects. My model looks like this:

targets <- glmer(state ~ ENV + HLH + HRI + LAB + SMO + Capital + 
(1 + rile + parties + rep + rep2 + gdppc + election| Country), 
data = df, family = binomial)

After centering and standardizing the predictors, the model converges fine after some hours. However, the output only gives me the Variance & Std.Dev. of the random effects, as well as the correlations among them, which makes sense for most multilevel analyses but not for my purposes. I am not so much interested in the variation of the intercepts and slopes of the model by country, but I rather want to know the direction of the impact of both action and state level characteristics, as well as whether it is significant. The summary of the glmer model gives me already that for the fixed effects. Is there any way I can get the estimates and the p-values for the random effects too?

If this cannot be done with Multilevel Regression Models or lme4 does not provide that output, is there any other model or software that would give such an output?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Normally you would increasingly (or decreasingly) add (or remove) random effects and test each model with the previous one with an ANOVA to test for random effect significance, that is if the new model improves upon the previous. As for the effect, whether it's positive or negative or something else, I do not think this can be done. $\endgroup$
    – user2974951
    Commented Dec 10, 2018 at 7:36

1 Answer 1

Reset to default
2
$\begingroup$

I don't think there is anyway to get p-values for the random effects, but you can use the est. of variance +/- ((Std.Dev/sqrt(n)) * 1.96) as a rough hand calculation to see if the interval includes zero - if it does then it is not significant at the 0.05 level - although this can be adjusted to different significance values. There are two schools of thought around random effects, as I understand it. There are those who believe that if a random effects should be retained regardless of signficance (purists), and those that remove them if they are insignificant. I am personally in the former - here are some good reasons for why.

As for calculating fixed effect p-values one can use must the same approach but calculating n is a little more complicated. There are three other approaches.

1) Fit the model without the effect of interest and then compare the two models using an anova (as they are nested) and this is an analogue for the desired p-value.

3) Use the predictmeans::predictmeans function - this outputs pairwise differences by default, as well as a few other informative plots.

3) The lmtest package might work for this - it was built to calculate these.

I can't check the later right now I am afraid as I am having a little trouble with R I am afraid....

Let me know if you are still having trouble after trying those out though!

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thanks for your answer! This is a great suggestion on how to calculate the p-value of random effects, but is there a way to calculate the estimate of those effects? ranef() gives the conditional modes of the random effects, but I would like to get an overall estimate for each one. $\endgroup$
    – Spl4t
    Commented Dec 10, 2018 at 8:29
  • $\begingroup$ If you print the summary of a glmer you should get the estimate of the random effect near the top of the output. If this answers your question then please give it a tick and a one up :) $\endgroup$
    – André.B
    Commented Dec 12, 2018 at 18:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.