0
$\begingroup$

I ran two models, a linear regression model and a linear mixed model, I did this because I was suspecting that there were some levels or hierarchy in my data, specifically in my subjects and scenarios. However I am not very familiar with the linear mixed model, I know that my subjects and the scenarios are the random effects, but I do not understand very well the output in this model. My first question is with the p value in the fixed effects, when I ran the regression model I got a p value of 0.03, but when I ran the mixed model I got 0.07, why this difference?. I also would like to know how I know how much the random effect affect the results like the p value, in what way I can say if it is worth it to run the LMM instead of other model?. these are my outputs: LMM:

MOD_would <- lme4::lmer(`puntaje likert` ~ nacion + (1|vignette)+(1|ResponseId), data = df_would,
                    control =  lme4::lmerControl(optimizer = "bobyqa", optCtrl=list(maxfun=100000)))

Random effects:
Predictors    Estimates       CI        p
(Intercept)    6.44        6.02 – 6.85  <0.001
nacion [ven]    -0.26    -0.55 – 0.02   0.072

Fixed effects:
            Estimate Std. Error t value
(Intercept)   6.4352     0.2095  30.722
nacionven    -0.2639     0.1461  -1.806

regression model:

mod_would2<- lm(`puntaje likert` ~ nacion , data = df_would)
summary(mod_wouldlm)

plot(mod_would2)


Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  6.43519    0.08912  72.205   <2e-16 ***
nacionven   -0.26389    0.12604  -2.094   0.0369 *  
$\endgroup$
1
  • $\begingroup$ These both are mixed models. The difference seems to be that the first only uses random intercepts and the second random slopes with respect to nacion. $\endgroup$
    – EdM
    Commented May 1, 2022 at 21:54

2 Answers 2

1
$\begingroup$

Given that you have run two different models, with differences in the underlying model equation, it is hardly surprising that the p-values for the resulting coefficient tests in those models would be different. Indeed, it would be an unbelievable coincidence if they were the same! Now, as regards the explanation for the difference you are observing here, what you can see is that there is stronger evidence for a non-zero association in the second model (MOD_would2). This occurs due to the residual association between your response variable and your explanatory variable nacion after accounting for the other model terms.

In terms of which model you should use, this is not determined by the p-values of the tests. Model choice in regression problems is largely determined by assessing appropriate diagnostic plots and added-variable plots of the models to determine whether the model assumptions are plausible representations of the data. Parsimonious analysis militates in favour of choosing the simplest model form that adequately represents the data, based on diagnostic analysis of the fitted model.

$\endgroup$
0
$\begingroup$

The p-value is an index of how strange the observed data would be if the null hypothesis was true, according to the statistical model. It depends on the data, the null hypothesis, and the model. You are using different models and so you should expect different p-values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.