I have the following experimental design, and am quite uncertain which statistical approach would be the best to investigate my data.

I have three experimental treatments of my participants and would like to know which influence has the treatment on the dependent variable and if this influence is mediated by a single latent variable. All variables, the dependent variable, the latent variable and the controls can be measured directly or indirectly by a questionnaire.

Please take a look at the following graphic ![enter image description here

First I thought that I could run multiple regressions, however, there will be a lot of regressions and I am unsure if I am runninng into errors of second order.

Do you have any recommendations or experiences with such designs and any idea how to handle that?

  • 6
    $\begingroup$ In general, we would need more information on what these variables are in order to recommend a salient technique for analyzing them. That said, I would recommend you look into "structural equation models". It is a class of methods for analyzing multivariate regressions incorporating latent constructs, and are often specified through path diagrams like this one. $\endgroup$ – Ryan Simmons Sep 26 '18 at 14:01
  • $\begingroup$ Hi Ryan, thank you! For the moment multi-group analysis in SEM seems to be a good way to taggle that problem! $\endgroup$ – JohnDoe2018 Sep 26 '18 at 14:45
  • 1
    $\begingroup$ @RyanSimmons That comment seems worth turning into an answer, even if it's a brief one. $\endgroup$ – mkt Sep 26 '18 at 15:23
  • $\begingroup$ @JohnDoe2018 I expanded my comment into a more detailed answer. One thing you may need to look into are your "Control variables"; you have them written as latent constructs with only two indicators each. In SEM parlance, this will likely be 'underidentified', meaning you won't have enough information in the model for reliable estimation. $\endgroup$ – Ryan Simmons Sep 26 '18 at 18:01
  • $\begingroup$ This is a very weird model. You have three treatments - treatments are usually considered to be causes. Yet you have arrows pointing to the treatments, implying that they are effects - can something influence your treatment group? $\endgroup$ – Jeremy Miles Sep 26 '18 at 18:40

In the absence of more detailed information, it seems like you are interested in "structural equation models", or SEM. This is a broad class of multivariate methods that incorporate latent constructs into a regression framework, often specified through path diagrams similar to yours. (That said, you actually have most of the arrows pointed in the wrong direction for a true path diagram. This makes it look like your treatment groups are the outcomes!)

In general, you use a path diagram that captures the theoretical relationships between the variables and/or constructs of interest, using this to build a variance-covariance matrix. The "structural" component of the SEM model is what captures the basic regression relationships, while the "measurement" model embeds a factor analysis-type structure into the model to capture the latent variables (indeed, both regression and factor analysis can be viewed as a special case of a more general SEM).

Within SEM, there are a number of different types of model. You allude to multi-group analysis in your comment. There are many others, and it's impossible to give more specific recommendations without knowing more about what all of these variables represent. Three words of caution:

1) Sample size. Typically, you need relatively large sample sizes to reliably fit an SEM, especially one with multiple latent constructs. One of the costs of SEM being a flexible and powerful model is that they are complicated, and the number of parameters can grow pretty quickly. If your sample size is small (e.g. <50), you are likely better served approaching the problem in a different way.

2) Model identification. There is a lot of literature on this issue in the SEM world. There's a bit of a learning curve to understanding it, but generally it becomes intuitive pretty quick, and you can often diagnose many identification problems just by glancing at a path diagram when you get enough experience with it. For example, looking at your path diagram, I immediately see a potential identification issue with your two "Control" factors, which only have two indicators loaded onto each one. This is likely to be underidentified; that is you don't have enough information to reliably estimate those factors, without modifying the model in some way (e.g. allowing the two factors to be correlated with one another).

3) Non-normal dependent variables. While there is a decent amount of literature on the issue of integrating the generalized linear model framework into SEM, and several software packages that will fit some of these models, it is still a more complicated and difficult model. In my experience, the software on these can be finicky for anything but the most simple models.

  • $\begingroup$ Thank you Ryan for that clarification - I think that will help experimental researcher a lot in their experimental design. Based on some literature scanning, I think the best practice would be to make a SEM for each treatment combination and then make a multigroup SEM analysis between the treatments. Do you think that will work? Furthermore, I updated the sketch of my planned experiment based on your comments to clarify my problem. $\endgroup$ – JohnDoe2018 Sep 28 '18 at 7:56
  • $\begingroup$ Unfortunately I don't know too much about "multigroup SEM analysis". So I don't know the best way to specify those types of models, or what advantages they have over more traditional SEM frameworks. Authors to look up are David A. Kenny, Kenneth Bollen, and Patrick Curran; each have a lot of published works on SEM models, and would be better sources than I (in fact, I learned SEM from Patrick Curran). $\endgroup$ – Ryan Simmons Sep 28 '18 at 15:51
  • $\begingroup$ Personally, I would simply specify the treatment combinations as dummy variables and fit a single SEM model over the set of dummy variables, with the appropriate coefficient paths between them and your mediators/outcome. But I don't know what the pros/cons of that approach are over the multigroup SEM framework. $\endgroup$ – Ryan Simmons Sep 28 '18 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.