25

This response will discuss possible models from a measurement perspective, where we are given a set of observed (manifest) interrelated variables, or measures, whose shared variance is assumed to measure a well-identified but not directly observable construct (generally, in a reflective manner), which will be considered as a latent variable. If you are ...


22

I take it the focus of the question is less on the theoretical side, and more on the practical side, i.e., how to implement a factor analysis of dichotomous data in R. First, let's simulate 200 observations from 6 variables, coming from 2 orthogonal factors. I'll take a couple of intermediate steps and start with multivariate normal continuous data that I ...


18

I would go for some papers by Múthen and Múthen, who authored the Mplus software, especially Múthen, B.O. (1984). A general structural equation model with dichotomous, ordered categorical and continuous latent indicators. Psychometrika, 49, 115–132. Muthén, B., du Toit, S.H.C. & Spisic, D. (1997). Robust inference using weighted least squares and ...


15

Yes, they are very different. Conceptually, a reflective measurement model happens when the indicators of a construct are considered to be caused by that construct. For example, an intelligence test: if you are more intelligent, you have a higher probability of getting the correct answer to a question. Hence your intelligence level is (theorized to) ...


14

The most commonly used correlational approach is called multi-trait scaling (MTS). The idea behind MTS is that items belonging to the same dimension are supposed to be more correlated to each other, as well as to their own scale, and that their correlations with items from other dimensions should be zero or at least of a lower magnitude. We generally ...


13

Exploratory factor analysis (EFA) is appropriate (psychometrically and otherwise) for examining the extent to which one may explain correlations among multiple items by inferring the common influence of (an) unmeasured (i.e., latent) factor(s). If this is not your specific intent, consider alternative analyses, e.g.: General linear modeling (e.g., multiple ...


13

The following two papers discuss cut-off values for reliability indices: Lance, C.E., Butts, M.M., & Michels, L.C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9 (2), 202-220. Henson, R.K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on ...


12

While only tangent to your goals at this point, if you continue on projects using latent variables I would highly suggest you read Denny Boorsboom's Measuring the Mind. Don't be fooled by the title, it is mainly a detailed essay on the logic of latent variables, and a large critique of classical test theory. I would say it is necessary reading if you are ...


11

The $\omega_h$ (hierarchical) coefficient gives the proportion of variance in scale scores accounted for by a general factor (1,2), usually from a second-order factor analysis. However, if any zero-order dimensions are reflected in such scales, $\omega_h$ will be less than Cronbach's $\alpha$ (which should only be used with unidimensional scales in any case)....


10

Two main causes: Small sample size. Even if the assumptions are met and the reliability is decent, an estimate computed from a particular sample can be negative, just as a sample mean is not equal to the population mean. Somewhat surprisingly, whereas experimental psychologists tend to be obsessed with statistical testing and are conditioned to ask a p-...


10

Although I feel a little sheepish contradicting both a "respected text" as well as another CV user, it seems to me that the Spearman-Brown formula is not affected by having items of differing difficulty. To be sure, the Spearman-Brown formula is usually derived under the assumption that we have parallel items, which implies (among other things) that the ...


9

I built on @Andy W's R-code and hope my changes are useful to someone else. I mainly changed it, so that it obeys the new syntax (no more opts) in ggplot2, so no more warnings adds the correlations as text now correlation text size reflects its effect size colour scheme shows the type of correlation (hetero/mono-trait/method). put the legend in the empty ...


9

The accepted answer does not give models where the response is bounded between [0,1]. There are IRT models for exactly the case where the response variable is continuous, but bounded in this way. For example, Samejima [1] describes exactly this case. These models are sometimes called Continuous Response Models (CRM), and the case they're addressing is ...


9

That's pretty normal. CFA is a much more stringent criterion than EFA. EFA attempts to describe your data, but CFA tests if the model is correct. One reason for non-convergence is low average correlations (but then I'd expect RMSEA to be better). The chi-square test is essentially a test that your residuals are equal to zero, and RMSEA, TLI and CFI are ...


9

Both classical test theory (CTT) and item response theory (IRT) can provide guidance as far as which items are contributing to the latent trait you wish to measure, and which do not. With CTT, consider 1) item difficulty, 2) item correlation to total score, 3) item variance, and 4) impact on internal consistency estimates (e.g., Cronbach's alpha) if the ...


8

Let me make a couple of points. First, if you just have 1 question, you don't technically have a likert scale, but just an ordinal rating. At any rate, I can't see as how there will be any meaningful difference. This is just a linear shift. This will neither make a difference whether you use an ordinal analysis like ordinal logistic regression or a Mann-...


8

There are two equivalent ways to express the parallel analysis criterion. But first I need to take care of a misunderstanding prevalent in the literature. The Misunderstanding The so-called Kaiser rule (Kaiser didn't actually like the rule if you read his 1960 paper) eigenvalues greater than one are retained for principal component analysis. Using the so-...


8

According to mathworld: de Moivre developed the normal distribution as an approximation to the binomial distribution, and it was subsequently used by Laplace in 1783 to study measurement errors and by Gauss in 1809 in the analysis of astronomical data (Havil 2003, p. 157). Some extensions to the normal distribution have been developed in psychometry, ...


8

You can see what it means by studying the formula: $$ \alpha = \frac{K}{K-1}\left(1-\frac{\sum \sigma^2_{x_i}}{\sigma^2_T}\right) $$ where $T=x_1 + x_2 + ... x_K$. $T$ is the total score of a test with $K$ items, each scores $x_i$, respectively. Unpack the formula, using what we know about the covariance of a sum of RV's. If the test items are ...


7

(You asked whether there is a statistical reason: I doubt it, but I'm guessing about other reasons.) Would there be cries of "moving the goalpost"? Students usually like to know when taking a test just how much each item is worth. They might be justified in complaining upon seeing, for example, that some of their hard-worked answers didn't end up ...


7

This was the recommended text on the course I took: P.B.Kline, Principles and Practice of Structural Equation Modeling, The Guilford Press. It is an introductory text, and not heavily mathematical. For a more mathematical, Bayesian, treatment, you could try: S-Y. Lee, Structural Equation Modeling: A Bayesian Approach, Wiley.


7

I must partially disagree with @MichaelChernick. While answers to a single Likert question (whether 0 to 5 or 1 to 6 or whatever) are clearly ordinal, usually there is a sum of several Likert scale items. At some point, the number of possible values becomes so high that it is essentially continuous. As you know (but the poster of the question may not) OLS ...


7

Cronbach’s alpha depends on the assumption that each indicator variable contributes equally to the factor, i.e., all (unstandardized) loadings must be the same (tau-equivalence). If this assumption is violated, true reliability will be underestimated. The second assumption for alpha is that the error variances of the indicators must be uncorrelated. In ...


7

A good approach to this kind of problem can be found in section 4 of the paper The Bayesian Image Retrieval System, PicHunter by Cox et al (2000). The data is a set of integer outcomes $A_1, ..., A_N$ where $N$ is the number of trials. In your case, there are 3 possible outcomes per trial. I will let $A_i$ be the index of the face that was left out. The ...


7

First, I would see if the doctors agree with each other. You can't analyze 50 doctors separately, because you'll overfit the model - one doctor will look great, by chance. You might try to combine confidence and diagnosis into a 10 point scale. If a doctors says that the patient doesn't have cancer, and they are very confident, that's a 0. If the doc says ...


6

I worked on this some the other day when you posted your same question to stack overflow. What I will provide won't be a finished solution, but hopefully it will give you enough ideas to finish the presentation on your own. This is what I could produce in SPSS, I have posted some code here using the same logic in R with ggplot2, but I'm not as familiar with ...


6

When you use the very simple structure method, you can choose between the VSS and the MAP criteria, that not ever reach to identical conclusions. Of course your VSS complexity 2 value is greater than the value of VSS complexity 1: you have to consider the greater value, so you can accept the complexity 2 solution (if I'm not mistaking about your data). ...


6

You can calculate the reliability of your items from the CFA. From your standardized solution, calculate: (L1+...Lk)*2/[(L1+...Lk)*2+(Var(E1)+...+Var(Ek))] This will give the composite reliability, which should be close to alpha. It's harder to have good fit if you have high alpha, and it's harder to have high alpha if you have good fit. The extreme ...


6

Cronbach's $\alpha$ tends to increase with the number of questions. If after deleting an item your Cronbach's $\alpha$ is the same, then you haven't gained any reliability by having that item. All other things being equal we tend to prefer shorter questionnaires... so we drop the item. One possible issue here is that the "Cronbach's $\alpha$ if deleted" ...


Only top voted, non community-wiki answers of a minimum length are eligible