# Using SEM to run one-way and two-way ANOVAs

I am interesseted in running an ANOVA using a structural-equation-model software. I have seen this question (Can we use SEM for doing ANOVA?) but it bears on a very simple one-way ANOVA with only two levels. I am seeking a more general response.

Because ANOVAs is a specific instance of a regression, and because SEM can be used to model any type of regression model, that should be doable, however, I am stucked with "singular matrix" problems.

Consider the following ficticious data for a simple one-way 3-level design with dummy-coded group labels in the first three columns, and the dependent variable in the fourth:

1   0   0   93
1   0   0   96
1   0   0   93
1   0   0   125
1   0   0   81
1   0   0   83
1   0   0   109
1   0   0   118
0   1   0   109
0   1   0   92
0   1   0   88
0   1   0   94
0   1   0   95
0   1   0   74
0   1   0   83
0   1   0   94
0   0   1   104
0   0   1   92
0   0   1   102
0   0   1   69
0   0   1   101
0   0   1   114
0   0   1   108
0   0   1   105


The ANOVA has no problem identifying the F ratio:

        DF  SumOfSq MeanSq  FRatio  PValue
Model   2   380.25  190.125 1.03914 0.371251
Error   21  3842.25 182.964
Total   23  4222.5


Likewise, with a multiple-regression analysis, I can get the $$R^2$$ from which I get the F ratio with $$\frac{R^2}{1-R^2} \times \frac{N-p}{p-1}$$ where $$N$$ is the total number of data (here 24) and $$p$$ is the number of predictors (here 3).

However, trying this with a structural-equation-modeling procedure (e.g., lavaan in R), I get an error, e.g.

lavaan ERROR: sample covariance matrix is not positive-definite

To that end, I used the instructions

library(lavaan)
my_model <- '
V4 ~ V1 +  V2 + V3 +1
'
my_fit <- sem(model = my_model, data = dta, estimator = "ML")
summary(my_fit, fit.measures = TRUE, standardized = TRUE, ci=TRUE, rsquare=TRUE)


I know that the third column is redundant with the previous two, but removing it leaves the error message unchanged.

How can the ANOVA be performed with an SEM platform? How would I generalize the solution to multiple between-subject factors (e.g., a 3 x 3 design)?

You seem to be pursuing two different questions:

1. What is the cause of your error(s)?

2. How can you use SEM to perform ANOVA-like analyses (incl. more complicated designs than one-way ANOVAs)

To be honest, I'm not sure about 1. I recreated your example dataset in excel, imported it, and all my code below (including the SEM snippets) run with no problems/complaints about not positive-definite sample covariance matrixes. I would: 1) double-check the way you are importing your data and making sure the variables are coded the way you would expect; 2) get rid of the redundant dummy coded variable; and 3) nix the superfluous "+1" at the end of your lavaan script object, but those are all sincerely guesses (and not exactly the intended focus of CV). It may also have something to do with the default settings that are applied in the sem wrapper of lavaan that you are using.

Regarding your more substantive question: How to do ANOVA-like analyses in SEM?

Disclaimer

I'd first offer a word of caution: regressions and ANOVAs are both amenable to the SEM framework, but in more complicated designs it's worth remembering that your typical ANOVA functions are doing more than they initially let on. Specifically, when you start analyzing designs with interactions (e.g., the 3x3 you propose), ANOVA functions are going to run behind the scenes a number of distinct models, and the summary output you get for main effects of $$X_1$$, and $$X_2$$, their combined main effects, and their interactions, are actually F-tests from a series of model comparisons of those models vs. the behind-the-scenes models. This is quite distinct from specifying such a design in regression, where the only model you get is the full one you specify (e.g., the 2-way interaction model), and the F-test compares it against an intercept-only model; if you want an alternative model comparison (e.g., interaction model vs. main effects model), you need to code it yourself.

SEM, such as when executed using lavaan's functionality can accommodate either approach (the many-models + comparison approach, or the you-get-what-you-specify approach), but the many-models approach requires a bit more careful and elaborate specification. In particular, once you get into interaction designs, it becomes increasingly crucial to understand how tests of interaction effects can be specified as very particular patterns of linear contrasts--a book like Maxwell and Delaney (2004) can be very useful for brushing up on the specific patterns of contrasts that might be interesting.

Finally, whereas in the ANOVA framework you are typically interesting in testing the "omnibus" effect of groups first, and then poking around to see which pairwise (or complex) comparisons are significant, in regression, the pairwise-level comparisons are extracted immediately through the estimation of your slopes for your dummy- (or effect- or contrast-)coded variables. You will see some parallel differences between the approaches for implementing ANOVA-like analyses in SEM.

SEM-based ANOVA: the Regression-Like Approach

Note: I label my dummy-coded variabes D_BA (B vs. A) and D_CA (C vs. A)

Just as you would specify a simple linear model in R:

reg.out <-lm(data = dat, Y ~ D_BA + D_CA)

So too can you specify the same sort of linear equation in lavaan:

lav1 <-' Y ~ D_BA + D_CA ' lav1.out <- cfa(lav1, data = dat) 

Running summary() on either output object produces nearly identical output; the slopes for the dummy variables are the same, though their standard errors differ (because lm uses OLS and lavaan defaults to ML for estimation).

SEM-based ANOVA: the ANOVA-Like Approach

In the ANOVA-like approach, we will utilize multi-group SEM/CFA, through which we will first be estimating different models in each of our three groups, and then forcing those models to be equal across groups, and evaluating the amount of model misfit introduced by that constraint. There are two ways of doing this in lavaan; they both work equally well in simple designs, but the latter is what you will want when evaluating more complicated interaction patterns.

Let lavaan automatically control group comparisons

lavaan makes it easy to control "families" of parameters across groups (the corresponding tutorial page is quite helpful to review this functionality).

First, we specify a multi-group model where the parameter of interest--in this case the intercept [or mean]--of each group is freely estimated across groups; this is done by default, as lavaan generally only constrains parameters that we tell it to:

lav2 <-' Y ~ 1 ' lav2.free.out <-cfa(lav2, data = dat, group = "Group") 

As you can see both the model specification and fitting code is pretty spartan. We just need an estimate of the intercept (~1) for Y, and the group argument in the cfa function indicates we want separate output for each level of "Group" (the column name of the three groups in my data frame).

If we are running a simple one-way design, or we want an "omnibus test" for the null hypothesis that all groups have equal means, we simply fit the same model with one additional argument in the cfa function:

lav2.eq.out <-cfa(lav2, data = dat, group = "Group", group.equal = c("intercepts"))  group.equal is the argument for forcing families of parameters to equality across groups. This is applied rather mindlessly, so (as you will see) it's not helpful for making more specific comparisons (or in more complicated designs, specifying particular contrasts corresponding to interaction tests).

With those two models fit (means freely estimated vs. means constrained to equivalence) we can use the anova function to compare the misfit introduce by the constraints:

anova(lav2.free.out, lav2.eq.out)

Note: you will get a different p-value here, in part because the test relies on a different sort of test statistic type ($$\chi^2$$) and in part because the way the test is conceptualized is different; it's testing the relative difference in fit between the two model's model-implied variance/covariance matrixes and the observed variance covariance matrix, as opposed to a ratio of variances as in an F-test.

Manually specify group comparisons

Instead of letting lavaan allow everything to be freely estimated or everything to be constrained, we might want to be more precise with the patterns we specify in order to facilitate particular comparisons.

Replicating the same analyses--but now with this "manual" approach--requires us to indicate for lavaan that we want unique parameters initially for each intercept. We do this by coding a vector of parameter labels (corresponding to the number of groups we have) and assigning (via *) these to the parameter of interest (the intercept, ~1). I have used a, b, and c, but these are arbitrary:

lav2.free.alt <-' Y ~ c(a,b,c)*1 ' lav2.free.alt.out <-cfa(lav2.free.alt, data = dat, group = "Group")  Once fitted, if you called the summary() output you would notice that the intercepts have the same estimated values as in lav2.free.out (when we let lavaan automatically free them) but now they also have the unique parameter labels we applied (a, b, and c).

To manually specify which parameters to constrain to equality, we simply change the labeling vector; groups with the same label will have their estimate constrained to equality. So to replicate the previous "omnibus test", all we do is assign all groups the same label (a, a, a):

lav2.eq.alt <-' Y ~ c(a,a,a)*1 ' lav2.eq.alt.out <-cfa(lav2.eq.alt, data = dat, group = "Group")  The output here would mirror lav2.eq.out, but once again, summary would also reveal our specified labels, and we would see that now every group's intercept has been given a label of "a".

Model comparison would then rely on the same anova function:

anova(lav2.free.alt.out, lav2.eq.alt.out) 

Where do interactions fit into all of this? You would first need a grouping variable consisting of different combinations of levels of your interacting variables (e.g., in a 3x3 of A:C and 1:3, A1, A2, A3, B1, B2, B3, C1, C2, C2). Then, it would come down to what patterns of interaction you would want to test, and specifying the appropriate contrast patterns in your labeling vector a la Maxwell and Delaney (2004) and comparing against an informative and interesting baseline model (e.g., perhaps a simpler main effect model).

References

Delaney, H. D., & Maxwell, S. E. (2004). Designing experiments and analyzing data. London, England: Psychology Press.

Three levels of treatment are represented with two dummy variables, because three dummy variables would create perfect collinearity. With three columns, the third is perfectly predicted by the other two. Deleting your third column (but retaining the rows with zeroes in both the first and second columns) resolves the problem. Deleting both the third column and the rows corresponding to the third treatment again creates perfect collinearity, as every case that is not the first treatment is necessarily the second treatment. As in regression, enter two predictor variables in your model to represent the three treatments. The third treatment, where both predictor variables have 0 values, becomes the reference condition. You could also perform the SEM analysis via a multiple populations approach, where each treatment defines a population, and the effect of each treatment is inferred from differences in the mean of the dependent variable across the three populations. I beg pardon for the earlier answer that was entirely off the mark.

• The explanation is not that clear. The correct dummy coding is to have 1s in the first column for the observations from the first group; 1s in the second column for the observations from the second group, and 0s in both columns for observations from the remaining group. Also it is necessary to set in the SEM model variances parameters on all three columns in addition to correlations from columns 1 and 2 to column 3. Mar 10, 2020 at 3:23
• I beg pardon--I answered a completely different question than the one which was asked. Mar 10, 2020 at 11:53