0
$\begingroup$

I have an unbalanced design with nested factors. In particular, the nested factor is obesity, with levels: of obese and nonobese. The other factor is group: women with syndrome, control women, and control men. The response let's say is a metabolite.

What I have done, is to set a linear model, with the following contrasts, due to the nature of the problem: the coefficients of the variates sum up to zero.

Then, in R one can achieve this with:

mdl <- lm(Y ~ obesity*group, 
    contrasts=list(obesity=contr. sum, 
    group=contr.sum)
car::Anova(mdl, type=3)

Then the output is

    Anova Table (Type III tests)

    Response: Y
                   Sum Sq Df   F value  Pr(>F)    
    (Intercept)    549.02  1 1117.2302 < 2e-16 ***
    obesity         0.19  1    0.3927 0.53446    
    group            2.09  2    2.1311 0.13199    
    obesity:group   3.21  2    3.2632 0.04866 *  
    Residuals       19.66 40                      
    ---
    Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

I tested for the assumptions of normality of the residuals and homogeneity, the model accomplishes this.

Now, as you can see, the interaction is significant, however, running TukeyHSD test, is not well suited because, from the help in R, it says:

" This is a generic function: the description here applies to the method for fits of class "aov" "

Thus, aov functions are the wrapper of anova(lm), but it does not take into consideration the contrasts, and it only does the sequential sum of squares

So I think, that the problem could be resolved with "marginal means"

The problem: I have no idea how to insert the formula with the error term.

I tested:

mdl2  <- aov_car(Y ~ obesity*group  + 
                 Error(patients)

Where patients are the subjects. I know that the error term, is the random mixed effect, (I think..)

Then I can run

emmeans(mdl2)

with the same output,

But I cannot figure out which is the significant group of interaction that the above results say.

So the question is why is significant the ANOVA but not the interaction ? Should I run pairwise t test with interaction ? I mean, interaction(obesity,group) and then run pairwise t.test ?

I have seen several posts about this issue, in particular, I think that the question is already answered on this package: https://cran.r-project.org/web/packages/emmeans/index.html

More than the data itself, it's a question of statistics

Improve this question
$\endgroup$
9
  • 1
    $\begingroup$ Since obesity has been dichotomized you have tremendous heterogeneity of body dimensions within each of the two obesity groups. That makes any result uninterpretable. $\endgroup$
    – Frank Harrell
    Commented Apr 26, 2022 at 11:51
  • $\begingroup$ I have like 300 variables on 4 blocks of data but firstly I want the univariate inference $\endgroup$
    – Edmond Geraud Aguilar
    Commented Apr 26, 2022 at 12:07
  • 1
    $\begingroup$ Please address the point I made. Doing lots of inference wrong does not make it right. Obesity is a matter of degree. It is not all-or-nothing. $\endgroup$
    – Frank Harrell
    Commented Apr 26, 2022 at 12:43
  • $\begingroup$ Dear Frank Harrell, I am aware of this situation. It's just a simple question I made. The purpose of the study is to do an integrative analysis. But I want to he familiarized with the data. If you know the answer, could you please answer me please ? $\endgroup$
    – Edmond Geraud Aguilar
    Commented Apr 26, 2022 at 12:54
  • 1
    $\begingroup$ Because you are using a continuous phenomenon inappropriately, resulting in severe lack of fit, loss of power, loss of information, ... nothing you will get from this will be interpretable. $\endgroup$
    – Frank Harrell
    Commented Apr 26, 2022 at 15:22

2 Answers 2

Reset to default
1
$\begingroup$

Whether you insist on keeping "obesity" as an all-or-none marker or, as Frank Harrell rightly suggests, you treat it as one or more continuous predictors (e.g., including both height and weight in some way), you have several potential sources of confusion in the way you are approaching this problem.

First, as the manual page for car::Anova says:

"Be careful of type-III tests... type-II tests are invariant with respect to (full-rank) contrast coding. If you don't understand this issue, then you probably shouldn't use Anova for type-III tests."

It's not clear how useful type-III tests will be, particularly with an unbalanced design and a potentially significant interaction. Your choice of type-III tests also seems to be forcing you to use sum contrasts. (I personally find the coefficients from sum contrasts harder to think about than those from the default treatment contrasts, although that might be my limitation and you might find otherwise.)

Second, it's not clear that you have more than one observation per participant for any one response variable. If you don't, then there's no "random effect" to deal with in univariate analysis, as you seem to be trying to do in your code. (If you have multiple response variables for each participant you would benefit from some type of multivariate analysis.)

Third, you ask "why is significant the ANOVA but not the interaction?" This might be a problem with terminology, but that's not what you show: the interaction term is significant (if barely) by the traditional p < 0.05 criterion.

Perhaps you are having problems identifying a particular "significant" pairwise comparison among the combinations of groups despite that overall finding of a "significant" interaction. That can happen in some circumstances, particularly when the overall "significance" is borderline as in the case you display.

Doing a set of pairwise t-tests as you propose would be an error unless you took into account the multiple-comparisons problem in some way. The TukeyHSD is only one way to do that; there are others.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thank you for the response, Firstly, I use these contrasts due to the fact that is not balanced, and on the other hand, I run out of degrees of freedom, that's why the usual contr.treatment does not help. Secondly, obviously is a multiple-comparison problem. TukeyHSD does not fit for me because, it makes SS sequentially. I know the naive approach of p-value, and the fallacy of. Let's suppose is not on the borderline of the p-value. So what type of posthoc test, using multiple comparisons, that takes into account the contrast should I run? $\endgroup$
    – Edmond Geraud Aguilar
    Commented Apr 26, 2022 at 19:05
  • $\begingroup$ Furthermore, as I said, this is to get familiar with the data, obviously, a multivariate strategy is better. And please, could we discuss this on chat? Let us assume that the problem the response is any continuous response with two factors as predictors, with this problem. $\endgroup$
    – Edmond Geraud Aguilar
    Commented Apr 26, 2022 at 19:19
  • 1
    $\begingroup$ @EdmondGeraudAguilar First, imbalance doesn't make contr.treatment invalid: it's invalid if you insist on type-III ANOVA, which I think is a mistake here. If you are doing all pairwise comparisons you don't need preliminary ANOVA at all. Second, the type of contrast doesn't affect the number of degrees of freedom. Third, the basic R TukeyHSD function might work on base aov with its sequential type-I ANOVA, but the emmeans package implements it for very general models. Or you could take your p-values from multiple t-tests and do a correction via the p.adjust() function. $\endgroup$
    – EdM
    Commented Apr 26, 2022 at 20:08
  • $\begingroup$ Ok, so, with unbalanced design, with contr.treatment, what I have read is that, sequential sum of squares does not lead to good results, so maybe tipe II? $\endgroup$
    – Edmond Geraud Aguilar
    Commented Apr 26, 2022 at 20:12
  • $\begingroup$ @EdmondGeraudAguilar it's true that type-I ANOVA with its sequential sums of squares doesn't work well with imbalance and contr.treatment, but you don't need to do ANOVA at all if you are going to do all pairwise comparisons. See this answer from the author of emmeans. You don't need to worry about that problem and can use any contrast type you wish. I have fit models with treatment contrasts and interactions and then used emmeans successfully to evaluate pairwise contrasts via the Tukey method. $\endgroup$
    – EdM
    Commented Apr 26, 2022 at 20:34
0
$\begingroup$

The purpose of my idea was to build a function that tests for every variable, on each block of data.

So, I did not realize that limma does exactly that (regardless of the experiment is well suited for factors such as obesity commented above).

Basically, the answer is [here ][1]

So, now it is time to check for assumptions [1]: https://stat.ethz.ch/pipermail/bioconductor/2011-June/040139.html

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.