Tagged Questions

Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

3
votes
2answers
940 views

Can I combine several ordinal questions to create an index and/or composite measure?

I am using a survey that contains several questions about various dimensions of performance for policy research institutes. Here, performance in the policy arena is unpacked into things like: ...
2
votes
0answers
68 views

Is it appropriate to assess internal consistency on related items measured with different response formats within the same instrument?

In my survey, I grouped 67 unique counseling techniques that I thought were related to one another (based on theory and conceptual similarity) into six "counseling domains" using fairly intuitive, ...
2
votes
1answer
325 views

IQ adaptive test items in 1pl, 2pl or 3pl IRT model?

Some adapative test systems (e.g. school assessment tools) use the 1pl IRT model, while others use the 2pl or the 3pl. When developing an adaptive IQ test, is there a rule of thumb about which model ...
1
vote
1answer
10k views

How to calculate validity of questionnaire in SPSS?

I have 3 types of questionnaires, A 15 item scale with each item on a 7-point Likert type scale. Six subscales each containing different number of items 21 subscales each containing 4 items. ...
1
vote
1answer
192 views

How to select next item in polytomous CAT / MIRT?

To make a Computerized Adaptive Test out of a sample of 20 dichotomous items a typical course of action would be to: 1) calibrate respondent data with a R package like mirt or ltm using Rasch, 2PL ...
3
votes
4answers
236 views

How can there be a standard deviation of true scores?

Dudek (1979) describes the following formula as "the standard deviation of true scores when the observed score is held constant" (p. 337): $SE_{est} = SD \sqrt{(reliability × (1 - reliability))}$ ...
6
votes
3answers
278 views

What is the reasoning behind the formulae for the different standard errors of measurement?

McManus (2012) discusses the three standard errors of measurement and provides the following definitions. $SE_{meas} = SD \sqrt{(1 - reliability)}$ The standard error of measurement is an estimate ...
3
votes
1answer
206 views

Why don't we square the reliability correlation when calculating reliable variance?

This question relates to situations where we apply some test to the same people at Time 1 and Time 2, and then calculate the correlation between scores at Time 1 and Time 2. I am reading the book ...
1
vote
0answers
85 views

how to analyze necessary/sufficient relationship?

Good day! I'm a second year psych grad student. From my fascination with formal logic, I wondered (just personal; not homework) about how to statistically establish necessary/sufficient ...
1
vote
0answers
111 views

What coefficient could I use to calculate the relative difficulty of a test in relation to others using only mean and population standard deviation?

I have a series of tests, all of them of different difficulty, and from each of them I get an average score and a population standard deviation; e.g: ...
5
votes
1answer
146 views

IRT/Rasch modeling with very large N

I want to fit a 1-parameter IRT model on a questionaire with 15 questions and about six million people. Considering the large N, standard errors aren't essential. It looks like the IRT world is sort ...
0
votes
1answer
80 views

EFA on R: Classifying Observations

I am newbie for R and Stats. I am using EFA in R [psych library] to identify the latent factors underlying my data. I got a reasonable amount of factors matching my purpose. My question is once I ...
1
vote
2answers
496 views

What factors do I have to consider when deciding if creating a composite score using four t-scores is defensible psychometrically?

Edit: I've conducted an item analysis on four variables - two quizzes and two psychological attributes, namely emotional intelligence and happiness. Its a within-subject design with students as ...
6
votes
1answer
307 views

EFA clearly supports one-factor, measure is internally consistent, but CFA has poor fit?

I am exploring the psychometric properties of a 10-item self-report measure. I have about 400 cases in two independent samples. The items are completed on 4-point Likert scales. An EFA clearly ...
2
votes
1answer
279 views

Comparing two logit or probit curves using a single parameter

I've conducted a psychological experiment on the same subject, under two different condition. For each condition I've collected the number of correct and wrong answer for each stimulus (number of ...
2
votes
2answers
374 views

How do I analyze a z-score to determine a difference?

My data are in SPSS. I teach in a nursing program. During their junior year, 68 students participated in a simulation scenario for heart failure. They were by rated by observers (0/1, did not do/did ...
5
votes
1answer
834 views

IRT in R: Does anyone know of an IRT item calibration function that can cope with NA's?

I have a dataset that looks like this. ...
4
votes
1answer
3k views

4 point and 5 point Likert scales in a new questionnaire

I'm looking to confirm whether having a 4 point Likert Scale for some constructs, and a 5 point Likert scale for other constructs will be suitable when developing a questionnaire? I have 7 constructs ...
0
votes
2answers
991 views

Error message from grm() in ltm: subscript out of bounds

I am analyzing a survey data set. One of the items in the survey is: ...
1
vote
0answers
425 views

Which correlation analysis is appropriate for binary and ordinal variables?

Violin players were asked to rate 5 violins from least to most rich on a [0 1], 0.05-increment scale, ...
3
votes
1answer
158 views

Omega H asymptotic

In the omega function in the psych package for R, we have several outputs including Hierarchical omega, Asymptotic omega and Total omega. I was reading about the outputs, but I find nothing about ...
1
vote
1answer
3k views

Understanding signficant interaction with non-significant main effects [duplicate]

Possible Duplicate: How to compute significant interaction estimates when main effect is not significant? I am an applied linguist and I am modelling responses to a vocabulary test taken by ...
2
votes
2answers
2k views

Likert scale and yes/no calculation of one score

My survey is based on observations and interviews. The observation questions are yes/no type questions and interview questions are 4 points. $1=$ never, $2=$ sometimes, $3=$ usually, and $4=$ ...
1
vote
0answers
562 views

How to validate a survey when using Likert scale?

I have developed a very simple survey to measure the attitudes of students toward science fairs. It makes use of the Likert Scale and Open Ended Questions. Do you have a recommendation for the type ...
2
votes
1answer
152 views

How to equate questionnaires with different response scales

I'm trying to help out a not-for-profit organization and made an error that I'm trying to fix. They've been doing a non scientific survey of their Board of Directors for a number of years regarding ...
4
votes
2answers
71 views

How to deal with the cases when participants gave rating of 2.5 or 4.5 on a 1-5 scale?

Because this was done during the interview, the participants just spoke out their rating rather than marked it done on the paper: they sometimes say "I think it is between 2 and 3" or "I give it 4 or ...
2
votes
1answer
253 views

How is Item Response Theory (IRT) called for continuous response?

I would like to model my problem using something similar to Item Response Theory - but my responses are not binary - they are continuous in [0;1]. How are these models/the research field called?
2
votes
1answer
176 views

Item information in IRT with item covariates (linear logistic test model)

Short question: How does the calculation and interpretation of IRT item information and test information change in the presence of item properties? Long question: There's a variation on IRT called ...
3
votes
1answer
158 views

Item information in IRT

According to item information curves, item information for a 2PL IRT model is $I(\theta)=a^2_i p_i(\theta) q_i(\theta)$ To determine $p_i(\theta)$ and $q_i(\theta)$, do you just use the observed ...
4
votes
1answer
668 views

Internal consistency reliability in item response theory models

How can internal consistency reliability of a test and of individual test items be quantified in Item Response Theory models? I know I can resort to Classical Test Theory, Cronbach's alpha, and other ...
0
votes
1answer
54 views

Test of symptom severity equality / symptom configuration equality

We have a questionnaire measuring 9 clinical symptoms. The sum scores simply adds all these symptoms up. Symptoms are ordered (0,1,2,3), and pretty strongly skewed (lots of 0s). 1) First, we want to ...
2
votes
2answers
2k views

Omega vs. alpha reliability

I wonder if someone can explain what is the main difference between omega and alpha reliabilities? I understand an omega reliability is based on hierarchical factor model as shown in the following ...
2
votes
0answers
124 views

Testing graded response model thresholds for significance?

I have a data set in which two raters have each rated N samples using a 5-point ordinal rating scale. My primary interest is in whether these two raters make significantly different use of those ...
4
votes
3answers
259 views

Are discrimination parameters in two-parameter IRT models only specific to items?

Discrimination parameters in two-parameter model from IRT are usually considered item parameters. But I've come to doubt it. Think about psychophysics; for example, detecting luminance. I don't ...
3
votes
1answer
1k views

Designing and validating a Likert-type scale with odd or even response categories

As part of an advisory team in my educational institution for undergraduate nurses, I (among others) have been assigned to design a questionnaire that will assess students' and academic staff's ...
3
votes
0answers
328 views

Dealing with poor fit in an Item Response Theory model

I'm studying an online course with about 3000 students who each took several quizzes and I'm trying to apply Item Response Theory (using the ltm package in R) to model the questions, determine which ...
1
vote
0answers
268 views

Obtaining factor scores from a factor analysis of mixed types (both ordinal and scale) variables

I have to analyze a data set having both ordinal and scale variables. What I was trying to do was to factor analyze the data. The data was in SPSS. But I think SPSS factor analyzes the data by ...
7
votes
1answer
288 views

What am I measuring when I apply a graded response model to the “Hunting of the Snark” dataset?

In another question, I asked about the statistical validity of StackExchange's "Hunting of the Snark" dataset, and whether or not we could draw any conclusions from its results. I measured a few ...
4
votes
3answers
452 views

$\chi^2$ tests to compare the fit of large samples logistic models

Does anyone know of any $\chi^2$ tests to compare the fit of logistic models which factor out the sample size? I'm dealing with a very large sample and I fear the significant $\chi^2$ test I get when ...
3
votes
1answer
211 views

How to apply published factor analysis loadings to sample data to compute factor scores?

I have data on 18 participants responding to a 30-item instrument. I also have a paper in which the researchers performed a principal axis factor analysis on the same survey where they report the item ...
3
votes
1answer
570 views

VSS criterion for the number of factors (in R's psych package)

I'm working from caracal's great example conducting a factor analysis on dichotomous data using R and I'm now struggling to understand the difference between VSS complexity 1 and VSS complexity 2 in ...
9
votes
2answers
3k views

How to reduce number of items using factor analysis, internal consistency, and item response theory in conjunction?

I am in the process of empirically developing a questionnaire and I will be using arbitrary numbers in this example to illustrate. For context, I am developing a psychological questionnaire aimed at ...
7
votes
4answers
3k views

Is 0 a valid value in a Likert scale?

I have carried out my pilot study on language learning motivation using a 6 point Likert scale but from 0 (strongly disagree) to 5 (very much agree). I noticed a colleague in his survey used 1 to 6. ...
9
votes
1answer
2k views

Looking for a step through an example of a factor analysis on dichotomous data (binary variables) using R

I have some dichotomous data, only binary variables, and my boss asked me to perform a factor analysis using the tetrachoric correlations matrix. I’ve previously been able to teach myself how to run ...
7
votes
1answer
925 views

How to perform inter-rater reliability with multiple raters, different raters per participant, and possible changes over time?

Participants were rated twice, with the 2 ratings separated by 3 years. For most participants the ratings were done by different raters, but for some (< 10%) the same rater performed both ratings. ...
5
votes
3answers
6k views

Is it acceptable to use Cronbach's alpha to assess reliability of questionnaire composed of categorical and conditional items?

Background: I have data on a questionnaire made up of categorical factual items. Some of them are binary and others have more than two categories. For example: ...
2
votes
1answer
83 views

What is correct terminology when describing the number of response levels of a Likert scale?

I know that the Likert scale is the sum (or some sort of combination) of all the responses of a survey, test etc. and the actual numeric responses are called response levels. Is this true of all ...
2
votes
1answer
72 views

Ordinal reliability for small examinee population

Biswas considers "a classical-type linear model of observed score ($X$), true score ($T$), and random error ($E$)" and asserts ... in test theory, interpretation of $\rho_{TX}$ has an intuitive ...
10
votes
2answers
8k views

How to transform ordinal data from questionnaire into proper interval data?

Are there any straightforward methods of transforming ordinal level data into interval level (just as there are for doing it the other way round)? And performable in Excel or SPSS? Having the data, ...
2
votes
1answer
278 views

Using total score from multi-scale instrument in structural equation modeling

I wonder if you can help me to get the right answer to a question about structural equation modeling. Imagine someone trying to validate a questionnaire using component factor analysis that ...