Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

2
votes
2answers
2k views

Likert scale and yes/no calculation of one score

My survey is based on observations and interviews. The observation questions are yes/no type questions and interview questions are 4 points. $1=$ never, $2=$ sometimes, $3=$ usually, and $4=$ ...
1
vote
0answers
398 views

How to validate a survey when using Likert scale?

I have developed a very simple survey to measure the attitudes of students toward science fairs. It makes use of the Likert Scale and Open Ended Questions. Do you have a recommendation for the type ...
2
votes
1answer
120 views

How to equate questionnaires with different response scales

I'm trying to help out a not-for-profit organization and made an error that I'm trying to fix. They've been doing a non scientific survey of their Board of Directors for a number of years regarding ...
4
votes
2answers
71 views

How to deal with the cases when participants gave rating of 2.5 or 4.5 on a 1-5 scale?

Because this was done during the interview, the participants just spoke out their rating rather than marked it done on the paper: they sometimes say "I think it is between 2 and 3" or "I give it 4 or ...
2
votes
1answer
179 views

How is Item Response Theory (IRT) called for continuous response?

I would like to model my problem using something similar to Item Response Theory - but my responses are not binary - they are continuous in [0;1]. How are these models/the research field called?
2
votes
1answer
151 views

Item information in IRT with item covariates (linear logistic test model)

Short question: How does the calculation and interpretation of IRT item information and test information change in the presence of item properties? Long question: There's a variation on IRT called ...
3
votes
1answer
116 views

Item information in IRT

According to item information curves, item information for a 2PL IRT model is $I(\theta)=a^2_i p_i(\theta) q_i(\theta)$ To determine $p_i(\theta)$ and $q_i(\theta)$, do you just use the observed ...
4
votes
1answer
360 views

Internal consistency reliability in item response theory models

How can internal consistency reliability of a test and of individual test items be quantified in Item Response Theory models? I know I can resort to Classical Test Theory, Cronbach's alpha, and other ...
0
votes
1answer
52 views

Test of symptom severity equality / symptom configuration equality

We have a questionnaire measuring 9 clinical symptoms. The sum scores simply adds all these symptoms up. Symptoms are ordered (0,1,2,3), and pretty strongly skewed (lots of 0s). 1) First, we want to ...
2
votes
2answers
1k views

Omega vs. alpha reliability

I wonder if someone can explain what is the main difference between omega and alpha reliabilities? I understand an omega reliability is based on hierarchical factor model as shown in the following ...
2
votes
0answers
103 views

Testing graded response model thresholds for significance?

I have a data set in which two raters have each rated N samples using a 5-point ordinal rating scale. My primary interest is in whether these two raters make significantly different use of those ...
4
votes
3answers
217 views

Are discrimination parameters in two-parameter IRT models only specific to items?

Discrimination parameters in two-parameter model from IRT are usually considered item parameters. But I've come to doubt it. Think about psychophysics; for example, detecting luminance. I don't ...
3
votes
1answer
691 views

Designing and validating a Likert-type scale with odd or even response categories

As part of an advisory team in my educational institution for undergraduate nurses, I (among others) have been assigned to design a questionnaire that will assess students' and academic staff's ...
2
votes
0answers
267 views

Dealing with poor fit in an Item Response Theory model

I'm studying an online course with about 3000 students who each took several quizzes and I'm trying to apply Item Response Theory (using the ltm package in R) to model the questions, determine which ...
1
vote
0answers
213 views

Obtaining factor scores from a factor analysis of mixed types (both ordinal and scale) variables

I have to analyze a data set having both ordinal and scale variables. What I was trying to do was to factor analyze the data. The data was in SPSS. But I think SPSS factor analyzes the data by ...
7
votes
1answer
272 views

What am I measuring when I apply a graded response model to the “Hunting of the Snark” dataset?

In another question, I asked about the statistical validity of StackExchange's "Hunting of the Snark" dataset, and whether or not we could draw any conclusions from its results. I measured a few ...
4
votes
3answers
369 views

$\chi^2$ tests to compare the fit of large samples logistic models

Does anyone know of any $\chi^2$ tests to compare the fit of logistic models which factor out the sample size? I'm dealing with a very large sample and I fear the significant $\chi^2$ test I get when ...
3
votes
1answer
197 views

How to apply published factor analysis loadings to sample data to compute factor scores?

I have data on 18 participants responding to a 30-item instrument. I also have a paper in which the researchers performed a principal axis factor analysis on the same survey where they report the item ...
3
votes
1answer
350 views

VSS criterion for the number of factors (in R's psych package)

I'm working from caracal's great example conducting a factor analysis on dichotomous data using R and I'm now struggling to understand the difference between VSS complexity 1 and VSS complexity 2 in ...
8
votes
2answers
2k views

How to reduce number of items using factor analysis, internal consistency, and item response theory in conjunction?

I am in the process of empirically developing a questionnaire and I will be using arbitrary numbers in this example to illustrate. For context, I am developing a psychological questionnaire aimed at ...
6
votes
3answers
2k views

Is 0 a valid value in a Likert scale?

I have carried out my pilot study on language learning motivation using a 6 point Likert scale but from 0 (strongly disagree) to 5 (very much agree). I noticed a colleague in his survey used 1 to 6. ...
7
votes
1answer
2k views

Looking for a step through an example of a factor analysis on dichotomous data (binary variables) using R

I have some dichotomous data, only binary variables, and my boss asked me to perform a factor analysis using the tetrachoric correlations matrix. I’ve previously been able to teach myself how to run ...
7
votes
1answer
836 views

How to perform inter-rater reliability with multiple raters, different raters per participant, and possible changes over time?

Participants were rated twice, with the 2 ratings separated by 3 years. For most participants the ratings were done by different raters, but for some (< 10%) the same rater performed both ratings. ...
5
votes
3answers
4k views

Is it acceptable to use Cronbach's alpha to assess reliability of questionnaire composed of categorical and conditional items?

Background: I have data on a questionnaire made up of categorical factual items. Some of them are binary and others have more than two categories. For example: ...
2
votes
1answer
76 views

What is correct terminology when describing the number of response levels of a Likert scale?

I know that the Likert scale is the sum (or some sort of combination) of all the responses of a survey, test etc. and the actual numeric responses are called response levels. Is this true of all ...
2
votes
1answer
70 views

Ordinal reliability for small examinee population

Biswas considers "a classical-type linear model of observed score ($X$), true score ($T$), and random error ($E$)" and asserts ... in test theory, interpretation of $\rho_{TX}$ has an intuitive ...
9
votes
2answers
6k views

How to transform ordinal data from questionnaire into proper interval data?

Are there any straightforward methods of transforming ordinal level data into interval level (just as there are for doing it the other way round)? And performable in Excel or SPSS? Having the data, ...
2
votes
1answer
231 views

Using total score from multi-scale instrument in structural equation modeling

I wonder if you can help me to get the right answer to a question about structural equation modeling. Imagine someone trying to validate a questionnaire using component factor analysis that ...
2
votes
0answers
270 views

Maximizing Cronbach alpha to select the number of variables that define a given construct

I can't seem to find this anywhere on the internet or in my books. I'm putting together a construct for an underlying variable. Is it acceptable to have a construct with two variables if that ...
4
votes
1answer
2k views

Item analysis for a likert-type questionnaire - item discrimination, point-biserial, factor analysis, cronbach's alpha, other diagnostics?

I'm working on a questionnaire that is intended to measure two constructs. There are presently 26 items in the questionnaire -- eleven for construct 1 and 15 for construct 2 -- and each item has five ...
1
vote
1answer
1k views

Interpreting AIC and BIC fit

I'm writing a CFA paper, and I have run into some trouble interpreting the AIC and BIC. This is my first paper using continuous variables, thus the first time I will be reporting these fit statistics ...
2
votes
0answers
346 views

Fixing error variance in a path analysis to model measurement error in scales using sem package

I want to construct a path analysis model that can account for measurement error in totally aggregated parcels, which refer to parcels where all of the items in a scale are summed or averaged. If I am ...
3
votes
3answers
1k views

A multitrait-multimethod matrix and data set

I am working my way through Multitrait-Multimethod Matrix in a psychometrics class. We're only required to be able to analyze them but I'd really like to be able to construct them. I think I am able ...
2
votes
2answers
220 views

Good starting book on psychometrics

I am looking for a nice concise statistics book that summarizes how to do the experimental designs properly, and maybe some algorithms people in the field have discovered. What are the best books for ...
6
votes
2answers
223 views

Reparameterizing the binomial link for psychometric data

Consider an experiment where a square and diamond appear on a screen, one before the other, and participants are required to judge which came first. Manipulating the time interval between the two ...
1
vote
3answers
1k views

What is the most effective way to combine different rating scales for the same data?

I have a collection of data which is inherently the same (would expect it to follow the same distribution) where some of it is rated with one scale of ordinal ratings and the rest of the data is rated ...
3
votes
2answers
352 views

Combining ratings from multiple raters of different accuracy

I have a data set of J raters who each give ratings to I objects (on say a scale of 1 to 5). My goal would be to construct some kind of overall rating for each i based on all of the raters' scores. ...
2
votes
3answers
185 views

Aggregate opinions to a single measure

I have a following situation in which 6 experts are asked to give opinion about a bottle of wine by rating it from 1 - 5 stars. In addition, somehow we have a measure of the informativeness of each ...
0
votes
0answers
361 views

Using Rasch model to explain relationships between a set of dependent and independent variables

My research study is in development economics. My data consist of more than one independent variables (continuous and categorical) as well as more than one dependent variables (categorical 5-point ...
15
votes
5answers
798 views

Introduction to structural equation modeling

I am asked by colleagues some help in this subject, that I don’t really know. They made hypotheses on the role of some latent variables in one study, and a referee asked them to formalize this in SEM. ...
6
votes
2answers
711 views

Advice on scientifically sound scale construction

I've been given a set of 20 Likert-items (ranging from 1-5, sample size n = 299) within the field of organizational research. The items are intended to measure a latent concept which is ...
2
votes
1answer
316 views

Predicting dependent variables where independent variables are samples with differing sample size

Is there a standard method of dealing with independent variables, where they are samples from a known distribution, and the sample sizes differ from sample to sample? I'll give an example of what I ...
1
vote
0answers
233 views

How is the proof that the Quartimax/Varimax-rotation converges?

Empirically the quartimax-/varimax-rotation has proven useful and it was always converging in my applications. But from my readings years ago I have a vague remembering, that the fact of a proof of ...
2
votes
1answer
320 views

Standard formula for quick calculation of scores

I have three datasets as follows: Dataset A can have a maximum score of 30. (Individual scores can be from 1 to 30) Dataset B can have a maximum score of 45. (Individual scores can be from 1 to 45) ...
4
votes
1answer
900 views

Which package to use for convergent and discriminant validity in R?

I am new to R and want to assess validity of some scales using R. Is there a package that I can use to test for convergent and discriminant validity in R?
2
votes
2answers
251 views

Removing human evaluator bias

I am working on machine translation evaluation, and am looking at ratings given by humans who are judging the quality of sentences produced by machine translation systems. The evaluators give fluency ...
8
votes
4answers
394 views

Why aren't all tests scored via item analysis/response theory?

Is there a statistical reason why item analysis/response theory isn't more widely applied? For instance, if a teacher gives a 25 question multiple choice test and finds that 10 questions were answered ...
1
vote
2answers
331 views

Rules of thumb for creating composite scales

Normally when I create an index I basically just take the or averages of the individual variables that comprise the index. But I'm wondering if one should / could not, take the product? E.g., a scale ...
5
votes
2answers
114 views

Sampling considerations in psychometric applications of item response theory

Background I have developed a recent interest in Item Response Theory and its applications. I am studying clinical psychology and am most interested in polytomous models aimed at modelling ...
14
votes
4answers
4k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...