7
$\begingroup$

Much like with regression, handling binary dependent variables in SEM requires special considerations. In particular, some of these are noted on Dave Garson's Structural Equation Modeling and include:

  1. Polychoric correlation. LISREL/PRELIS uses polyserial, tetrachoric, and polychoric correlations to create the input correlation matrix, combined with ADF estimation (see below), for variables which cannot be assumed to have a bivariate normal distribution.

    • Sample size issue. ADF [Asymptotically distribution-free] estimation in turn requires a very large sample size. Yuan and Bentler (1994) found satisfactory estimates only with a sample size of at least 2,000 and preferably 5,000. Violating this requirement may introduce problems greater than treating ordinal data as interval and using ML estimation. This is also a reason cited for preferring the Bayesian estimation approach to ordinal data taken by Amos since Bayesian estimation can handle smaller samples than ML or ADF.

I'm currently trying to use the package sem in R to test my model, and the author of the model suggests using polychoric correlations on R-help. The problems are:

  1. I don't know what estimation method is being used with these correlations (i.e., ADF or ML).
  2. My sample size is small (N = 173).
  3. I'm not familiar with how to interpret polychoric associations (in the case that it is appropriate for me to use them). All the other variables in my model are continuous in nature.

Any help and/or links would be greatly appreciated. I'm also considering using other software like OpenMX, but I'm still reading about how it handles binary data. Help with what other software I might want to use would also be appreciated.

$\endgroup$
  • $\begingroup$ Joaqin, welcome to the website. You might want to raise this question on SEMNET, a mailing list on basic SEM education. $\endgroup$ – StasK Mar 19 '12 at 2:46
7
$\begingroup$

Did you read the original Olsson (1979) paper? I believe it still provides the best description of what polychoric correlations are (although I've probably skimmed only 10% of the existing literature, I have to admit; at some point, it just gets too repetitive of the limited number of ideas though). Polychoric correlations are ML estimates of the correlations of the underlying normal distribution, so you interpret them just as you would Pearson moment correlations with continuous data. Given the ML origins of polychoric correlations, I never understood the advice to use ADF or other least squares methods with them to obtain model parameter estimates, although I do understand that say diagonally weighted least squares (don't know if John Fox implemented them in sem though), while being less asymptotically efficient, don't need as much auxiliary information for estimation purposes.

There is no magic sample size number, like, you hit 2000 and -- BOOM! -- everything starts working. In my simulations (and I've done a few petaflops this way and that way for my papers), I've seen both cases when asymptotic results worked perfectly fine with $N=200$ and failed to work with $N=5000$. In the most peculiar cases, for the same method and distribution of the underlying data, some asymptotic aspects, such as confidence interval coverage say, would be OK for $N=300$, while others, like $\chi^2$ distribution of a test statistic, would not work until you have $N=1000$. So I am highly skeptical of any sample size advice, and would rather recommend to run a simulation addressing your particular sample size, model complexity and magnitude of the errors. The first paper to bash ADF (Hu, Bentler and Kano (1992)) used an insane degree of overidentification, something like 30 variables in the model, which translates to 400 degrees of freedom, and a sample size of 50. ADF wouldn't even begin to work in these circumstances, as it won't be able to invert the matrix of the fourth moments which will be rank-deficient. And to get 400 degrees of freedom for the test statistic with the sample size below 1000 is a high expectation, too.

So I understand the healthy skepticism that you are demonstrating, but there is simply nothing you can do in your situation about it. Just run polycor to get the correlation estimates, feed them to sem, and that would be it -- there is little you can do to produce a much better analysis.

If you were a Stata user, I would immediately recommend gllamm package, but I am not sure whether a direct analogue of it exists in R.

$\endgroup$
  • $\begingroup$ Thank you! The results are consistent with what I was expecting given pearson correlations, so I will use them. I've seen your posts elsewhere and it seems like you are very familiar with this topic. As such, I was hoping you could help me with a follow up question. Namely, do you know how I could compute linear regression in R using polyserial correlations? I'm unable to find any information related to this topic online. In particular, I would like to obtain the residuals associated with this regression. $\endgroup$ – Joaquin Aguilar Mar 20 '12 at 3:27
  • $\begingroup$ I think it is neither possible nor interesting. Does it make sense to look at residuals in an ordinal logit/probit regression? They are hard to conceptualize. $\endgroup$ – StasK Mar 23 '12 at 4:32
  • $\begingroup$ The residuals are interesting insofar as they allow me to use them in SEM and claim that for the different variables in the model, I controlled for other variables even though they are not included in the actual SEM model. Thank you. $\endgroup$ – Joaquin Aguilar Mar 26 '12 at 16:14
4
$\begingroup$

@StasK is correct in stating that the polychoric correlations can be interpreted similarly to pearson correlations, however, it sounds as if you are attempting to build a latent variable model and not simply interpret the correlation matrix, so i suggest not worry about direct interpretation. suffice to say that polychoric correlations are appropriate for binary indicators.

the problem with binary indicators really stems from the fact that you're talking about (severely) non-normally distributed variables. this is a problem that normal theory estimators, such as ML and GLS, struggle to overcome - ML and GLS typically estimate inflated $\chi^2$ model-fit indices as well as under-estimating parameter variance, both leading to inflated type-I errors. nevermind the fact that you also have a small sample size.

given these issues, the weighted least squares with mean and variance correction (WLSMV) has been shown to be the most appropriate estimator for use with binary indicators. unfortunately, this estimator is only available in the Mplus software. Other than Mplus, the fa.poly function in the R package psych implements a WLS estimator which still runs into issues with sample size but is preferrable over ADF or ML estimation.

for a good overview on the topic of categorical data in SEM (and, really, any latent variable model), i recommend the accessible chapter by Finney and DiStefano (2006).

...though you mentioned a continuous indicator, depending on the model you're trying to estimate, you may give item response theory (IRT) models a look. under certain conditions, they are seen to be equivalent to CFA/SEM models but differ in the estimation approach. Finch (2010) does a good job of illustrating the IRT/CFA equivalence.

Finch, H. (2010). Item parameter estimation for the MIRT model: Bias and precision of confirmatory factor-analysis based models. Applied Psychological Measurement, 34, 10- 26.

Finney, S. J., & DiStefano, C. (2006). Nonnormal and categorical data in structural equation models. In G.R. Hancock & R.O. Mueller (Eds.). A second course in structural equation modeling (pp. 269-314). Greenwich, CT: Information Age.

$\endgroup$
  • $\begingroup$ Thank you! I spoke with my advisor and he also suggested I try Mplus, so I will in the near future, but to the extent possible, I always prefer to do statistics in R. I don't know anything about item response theory, but I'm really interested in finding more about it, although this will have to wait until I finish my dissertation. Thank you for the links! $\endgroup$ – Joaquin Aguilar Mar 20 '12 at 3:30
  • $\begingroup$ If you are a curious grad student with a lot of time in your hands, you can just program everything up -- or join a project like OpenMx or lavaan. Figuring things out and writing good code takes a lot of time, but the pay off in terms of figuring out what each method actually does, and how it does it, simply cannot be compared with any other research experience. I learned polychoric correlations by programming up the estimator in Stata :) $\endgroup$ – StasK Mar 23 '12 at 4:35
  • $\begingroup$ Completely agree with you. I have a bachelors in computer science, so I really like to code my statistics. Unfortunately, time is scarce since I'm in the finishing stages of my dissertation, but once I have more time I definetely want to learn more about OpenMX, which seems to give people more flexibility. $\endgroup$ – Joaquin Aguilar Mar 26 '12 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.