# Tag Info

25

This response will discuss possible models from a measurement perspective, where we are given a set of observed (manifest) interrelated variables, or measures, whose shared variance is assumed to measure a well-identified but not directly observable construct (generally, in a reflective manner), which will be considered as a latent variable. If you are ...

22

I take it the focus of the question is less on the theoretical side, and more on the practical side, i.e., how to implement a factor analysis of dichotomous data in R. First, let's simulate 200 observations from 6 variables, coming from 2 orthogonal factors. I'll take a couple of intermediate steps and start with multivariate normal continuous data that I ...

16

Yes, they are very different. Conceptually, a reflective measurement model happens when the indicators of a construct are considered to be caused by that construct. For example, an intelligence test: if you are more intelligent, you have a higher probability of getting the correct answer to a question. Hence your intelligence level is (theorized to) ...

13

Exploratory factor analysis (EFA) is appropriate (psychometrically and otherwise) for examining the extent to which one may explain correlations among multiple items by inferring the common influence of (an) unmeasured (i.e., latent) factor(s). If this is not your specific intent, consider alternative analyses, e.g.: General linear modeling (e.g., multiple ...

13

The following two papers discuss cut-off values for reliability indices: Lance, C.E., Butts, M.M., & Michels, L.C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9 (2), 202-220. Henson, R.K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on ...

11

The $\omega_h$ (hierarchical) coefficient gives the proportion of variance in scale scores accounted for by a general factor (1,2), usually from a second-order factor analysis. However, if any zero-order dimensions are reflected in such scales, $\omega_h$ will be less than Cronbach's $\alpha$ (which should only be used with unidimensional scales in any case)....

10

The accepted answer does not give models where the response is bounded between [0,1]. There are IRT models for exactly the case where the response variable is continuous, but bounded in this way. For example, Samejima  describes exactly this case. These models are sometimes called Continuous Response Models (CRM), and the case they're addressing is ...

10

Two main causes: Small sample size. Even if the assumptions are met and the reliability is decent, an estimate computed from a particular sample can be negative, just as a sample mean is not equal to the population mean. Somewhat surprisingly, whereas experimental psychologists tend to be obsessed with statistical testing and are conditioned to ask a p-...

10

Although I feel a little sheepish contradicting both a "respected text" as well as another CV user, it seems to me that the Spearman-Brown formula is not affected by having items of differing difficulty. To be sure, the Spearman-Brown formula is usually derived under the assumption that we have parallel items, which implies (among other things) that the ...

9

I built on @Andy W's R-code and hope my changes are useful to someone else. I mainly changed it, so that it obeys the new syntax (no more opts) in ggplot2, so no more warnings adds the correlations as text now correlation text size reflects its effect size colour scheme shows the type of correlation (hetero/mono-trait/method). put the legend in the empty ...

9

That's pretty normal. CFA is a much more stringent criterion than EFA. EFA attempts to describe your data, but CFA tests if the model is correct. One reason for non-convergence is low average correlations (but then I'd expect RMSEA to be better). The chi-square test is essentially a test that your residuals are equal to zero, and RMSEA, TLI and CFI are ...

9

Both classical test theory (CTT) and item response theory (IRT) can provide guidance as far as which items are contributing to the latent trait you wish to measure, and which do not. With CTT, consider 1) item difficulty, 2) item correlation to total score, 3) item variance, and 4) impact on internal consistency estimates (e.g., Cronbach's alpha) if the ...

9

There are two equivalent ways to express the parallel analysis criterion. But first I need to take care of a misunderstanding prevalent in the literature. The Misunderstanding The so-called Kaiser rule (Kaiser didn't actually like the rule if you read his 1960 paper) eigenvalues greater than one are retained for principal component analysis. Using the so-...

8

Let me make a couple of points. First, if you just have 1 question, you don't technically have a likert scale, but just an ordinal rating. At any rate, I can't see as how there will be any meaningful difference. This is just a linear shift. This will neither make a difference whether you use an ordinal analysis like ordinal logistic regression or a Mann-...

8

According to mathworld: de Moivre developed the normal distribution as an approximation to the binomial distribution, and it was subsequently used by Laplace in 1783 to study measurement errors and by Gauss in 1809 in the analysis of astronomical data (Havil 2003, p. 157). Some extensions to the normal distribution have been developed in psychometry, ...

8

You can see what it means by studying the formula: $$\alpha = \frac{K}{K-1}\left(1-\frac{\sum \sigma^2_{x_i}}{\sigma^2_T}\right)$$ where $T=x_1 + x_2 + ... x_K$. $T$ is the total score of a test with $K$ items, each scores $x_i$, respectively. Unpack the formula, using what we know about the covariance of a sum of RV's. If the test items are ...

8

As far as I can tell you are describing a partially crossed design. The good news is that this is one of Doug Bates's main development goals for lme4: efficiently fitting large, partially crossed linear mixed models. Disclaimer: I don't know that much about Rasch models nor how close a partially nested model like this gets to it: from a brief glance at ...

7

I must partially disagree with @MichaelChernick. While answers to a single Likert question (whether 0 to 5 or 1 to 6 or whatever) are clearly ordinal, usually there is a sum of several Likert scale items. At some point, the number of possible values becomes so high that it is essentially continuous. As you know (but the poster of the question may not) OLS ...

7

Cronbach’s alpha depends on the assumption that each indicator variable contributes equally to the factor, i.e., all (unstandardized) loadings must be the same (tau-equivalence). If this assumption is violated, true reliability will be underestimated. The second assumption for alpha is that the error variances of the indicators must be uncorrelated. In ...

7

A good approach to this kind of problem can be found in section 4 of the paper The Bayesian Image Retrieval System, PicHunter by Cox et al (2000). The data is a set of integer outcomes $A_1, ..., A_N$ where $N$ is the number of trials. In your case, there are 3 possible outcomes per trial. I will let $A_i$ be the index of the face that was left out. The ...

7

First, I would see if the doctors agree with each other. You can't analyze 50 doctors separately, because you'll overfit the model - one doctor will look great, by chance. You might try to combine confidence and diagnosis into a 10 point scale. If a doctors says that the patient doesn't have cancer, and they are very confident, that's a 0. If the doc says ...

6

When you use the very simple structure method, you can choose between the VSS and the MAP criteria, that not ever reach to identical conclusions. Of course your VSS complexity 2 value is greater than the value of VSS complexity 1: you have to consider the greater value, so you can accept the complexity 2 solution (if I'm not mistaking about your data). ...

6

You can calculate the reliability of your items from the CFA. From your standardized solution, calculate: (L1+...Lk)*2/[(L1+...Lk)*2+(Var(E1)+...+Var(Ek))] This will give the composite reliability, which should be close to alpha. It's harder to have good fit if you have high alpha, and it's harder to have high alpha if you have good fit. The extreme ...

6

Cronbach's $\alpha$ tends to increase with the number of questions. If after deleting an item your Cronbach's $\alpha$ is the same, then you haven't gained any reliability by having that item. All other things being equal we tend to prefer shorter questionnaires... so we drop the item. One possible issue here is that the "Cronbach's $\alpha$ if deleted" ...

5

Looks like I'm quite late to the game here, but the mirt package package can estimate WLE scores for dichotomous and polytomous models. You start by fitting, say, a graded response model to your data (or whatever your model may be, PCM, generalised PCM, nominal, rating scale, etc; see ?mirt for the possible options) then compute either a table summary of the ...

5

Q1. 4 or 5 point scale (strongly disagree to strongly agree with or without a neutral midpoint) A1. I think the use of even or odd number of scale points is not a matter that has a definitive answer. There are arguments on both sides of this question. Since you want a yes-no answer, the 4 point scale may be better suited to your purpose than a scale with ...

5

In following up on @caracal's reference suggestions, I found an almost-direct answer (no, these two rating systems are not equivalent if presented as number options to respondents) from Schwarz, Knäuper, Hippler, Noelle-Neumann, and Clark (1991). They present data on responses to the question, "How successful have you been in life, so far?" One version gave ...

5

You can compute test information curves from your IRT parameter estimates. These curves give you the precision of the test at each $\theta$ of the latent trait. The information $I$ can be transformed into the standard error of estimate $SEE$, which is a direct estimate of the reliability of the test at that $\theta$: $SEE = 1 / \sqrt{I}$. The metric of ...

5

If you have a continuous indicator, then you would use factor analysis. Think of FA as linear regression and IRT it's logistic regression brother.

5

As I stated in the comments above, missing data can be handled by either the ltm or mirt package when the data is MCAR. Here is an example of how to use both on a dataset with missing values: > library(ltm) > library(mirt > set.seed(1234) > dat <- expand.table(LSAT7) > dat[sample(1:(nrow(dat)*ncol(dat)), 150)] <- NA > head(dat) ...

Only top voted, non community-wiki answers of a minimum length are eligible