# lsmeans output has identical p-values for different groups

I'm looking at some data with an lme4 model. My variables are sex, treatment with two levels, element with three levels (each of them a nutrient: carbs, fat, protein) and age (young, mature).

looks like this:

lmer(formula = value ~ element + sex + condition + ageGroup + (element|subjectNumber)


this is the output:

                    Sum Sq Mean Sq NumDF   DenDF F.value    Pr(>F)
element            4130068 2065034     2  54.385 140.742 < 2.2e-16 ***
sex                 219840  219840     1  43.835  14.983 0.0003568 ***
condition            80857   80857     1 170.201   5.511 0.0200495 *
ageGroup             10939   10939     1  42.765   0.746 0.3927161
element:sex         121265   60633     2  54.000   4.132 0.0213836 *
element:condition    94784   47392     2 169.332   3.230 0.0420081 *
element:ageGroup    128680   64340     2  54.281   4.385 0.0171648 *
sex:ageGroup         10423   10423     1  55.313   0.710 0.4029552
sex:condition        36221   36221     1 169.519   2.469 0.1180025
condition:ageGroup    2311    2311     1 170.019   0.157 0.6919849


From here I concluded that the treatment had an effect, independently of age and sex. (main effect for condition sign. plus sign. element:condition)

My boss wants me to run post-hoc comparisons for each sex (male-carbs-treat1 vs male-carbs-treat2; male-fat-treat1 vs male-fat-treat2; etc). I understand this is not "allowed", but I had to do it anyway.

I piped (%>%) the lmer model into lsmeans:

lsmeans(pairwise~condition*element|sex, adjust="mvt")

\$contrasts
sex = male:
contrast                                      estimate       SE     df t.ratio p.value
Treatment1,carbs_kcal - Treatment2,carbs_kcal      89.731996 25.75586 171.88   3.484  0.0082
Treatment1,carbs_kcal - Treatment1,fat_kcal       130.308762 36.44096  71.95   3.576  0.0081
Treatment1,carbs_kcal - Treatment2,fat_kcal       147.012243 36.49856  72.45   4.028  0.0019
Treatment1,carbs_kcal - Treatment1,protein_kcal   483.768384 38.90480  78.34  12.435  <.0001
Treatment1,carbs_kcal - Treatment2,protein_kcal   487.977078 38.94262  78.68  12.531  <.0001
Treatment2,carbs_kcal - Treatment1,fat_kcal        40.576766 36.50478  72.47   1.112  0.8828
Treatment2,carbs_kcal - Treatment2,fat_kcal        57.280247 36.52338  72.25   1.568  0.6373
Treatment2,carbs_kcal - Treatment1,protein_kcal   394.036388 38.95686  78.69  10.115  <.0001
Treatment2,carbs_kcal - Treatment2,protein_kcal   398.245082 38.98118  78.69  10.216  <.0001
Treatment1,fat_kcal - Treatment2,fat_kcal          16.703481 25.72907 172.04   0.649  0.9878
Treatment1,fat_kcal - Treatment1,protein_kcal     353.459622 33.90592 118.31  10.425  <.0001
Treatment1,fat_kcal - Treatment2,protein_kcal     357.668316 33.95106 118.85  10.535  <.0001
Treatment2,fat_kcal - Treatment1,protein_kcal     336.756141 33.96071 118.75   9.916  <.0001
Treatment2,fat_kcal - Treatment2,protein_kcal     340.964835 33.98515 118.67  10.033  <.0001
Treatment1,protein_kcal - Treatment2,protein_kcal   4.208694 25.69036 170.68   0.164  1.0000

sex = female:
contrast                                      estimate       SE     df t.ratio p.value
Treatment1,carbs_kcal - Treatment2,carbs_kcal      89.731996 25.75586 171.88   3.484  0.0082
Treatment1,carbs_kcal - Treatment1,fat_kcal       157.740996 38.34494  68.53   4.114  0.0015
Treatment1,carbs_kcal - Treatment2,fat_kcal       174.444477 38.63288  69.80   4.515  0.0004
Treatment1,carbs_kcal - Treatment1,protein_kcal   393.947473 40.99898  75.48   9.609  <.0001
Treatment1,carbs_kcal - Treatment2,protein_kcal   398.156167 41.22497  76.75   9.658  <.0001
Treatment2,carbs_kcal - Treatment1,fat_kcal        68.009000 38.67670  69.90   1.758  0.5141
Treatment2,carbs_kcal - Treatment2,fat_kcal        84.712481 38.92567  70.53   2.176  0.2707
Treatment2,carbs_kcal - Treatment1,protein_kcal   304.215477 41.32524  76.79   7.361  <.0001
Treatment2,carbs_kcal - Treatment2,protein_kcal   308.424171 41.53682  77.76   7.425  <.0001
Treatment1,fat_kcal - Treatment2,fat_kcal          16.703481 25.72907 172.04   0.649  0.9878
Treatment1,fat_kcal - Treatment1,protein_kcal     236.206477 35.59762 114.57   6.635  <.0001
Treatment1,fat_kcal - Treatment2,protein_kcal     240.415171 35.85408 116.40   6.705  <.0001
Treatment2,fat_kcal - Treatment1,protein_kcal     219.502996 35.92219 115.78   6.111  <.0001
Treatment2,fat_kcal - Treatment2,protein_kcal     223.711690 36.15696 117.04   6.187  <.0001
Treatment1,protein_kcal - Treatment2,protein_kcal   4.208694 25.69036 170.68   0.164  1.0000


Is it normal for the relevant p-values (carbs vs carbs, fat vs fat, etc) to be exactly the same in both groups? I would really like to know if a) I'm doing a mistake somewhere and b) if this behavior is normal.

• This is normal if you don't have an interaction between condition and sex. The model format you show in the beginning does not include such an interaction whereas the model summary shows an interaction ... Something is not right there ... Commented Nov 30, 2016 at 11:08
• "I understand this is not 'allowed'" Why shouldn't it be? As long as you adjust the p-values appropriately ... Commented Nov 30, 2016 at 11:09
• @Roland: you are right about the interactions, I forgot to paste them but my code has them element:sex + element:condition + element:ageGroup + sex:condition By "not allowed" I mean it is my understanding that one shouldn't run post-hoc tests on non-significant interactions Commented Nov 30, 2016 at 11:39
• You might need to provide a reproducible example. Commented Nov 30, 2016 at 11:45