Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

3
votes
3answers
2k views

A multitrait-multimethod matrix and data set

I am working my way through Multitrait-Multimethod Matrix in a psychometrics class. We're only required to be able to analyze them but I'd really like to be able to construct them. I think I am able ...
2
votes
2answers
274 views

Good starting book on psychometrics

I am looking for a nice concise statistics book that summarizes how to do the experimental designs properly, and maybe some algorithms people in the field have discovered. What are the best books for ...
6
votes
2answers
238 views

Reparameterizing the binomial link for psychometric data

Consider an experiment where a square and diamond appear on a screen, one before the other, and participants are required to judge which came first. Manipulating the time interval between the two ...
1
vote
3answers
1k views

What is the most effective way to combine different rating scales for the same data?

I have a collection of data which is inherently the same (would expect it to follow the same distribution) where some of it is rated with one scale of ordinal ratings and the rest of the data is rated ...
3
votes
2answers
417 views

Combining ratings from multiple raters of different accuracy

I have a data set of J raters who each give ratings to I objects (on say a scale of 1 to 5). My goal would be to construct some kind of overall rating for each i based on all of the raters' scores. ...
2
votes
3answers
194 views

Aggregate opinions to a single measure

I have a following situation in which 6 experts are asked to give opinion about a bottle of wine by rating it from 1 - 5 stars. In addition, somehow we have a measure of the informativeness of each ...
0
votes
0answers
412 views

Using Rasch model to explain relationships between a set of dependent and independent variables

My research study is in development economics. My data consist of more than one independent variables (continuous and categorical) as well as more than one dependent variables (categorical 5-point ...
16
votes
5answers
952 views

Introduction to structural equation modeling

I am asked by colleagues some help in this subject, that I don’t really know. They made hypotheses on the role of some latent variables in one study, and a referee asked them to formalize this in SEM. ...
6
votes
2answers
918 views

Advice on scientifically sound scale construction

I've been given a set of 20 Likert-items (ranging from 1-5, sample size n = 299) within the field of organizational research. The items are intended to measure a latent concept which is ...
2
votes
1answer
336 views

Predicting dependent variables where independent variables are samples with differing sample size

Is there a standard method of dealing with independent variables, where they are samples from a known distribution, and the sample sizes differ from sample to sample? I'll give an example of what I ...
1
vote
0answers
254 views

How is the proof that the Quartimax/Varimax-rotation converges?

Empirically the quartimax-/varimax-rotation has proven useful and it was always converging in my applications. But from my readings years ago I have a vague remembering, that the fact of a proof of ...
2
votes
1answer
374 views

Standard formula for quick calculation of scores

I have three datasets as follows: Dataset A can have a maximum score of 30. (Individual scores can be from 1 to 30) Dataset B can have a maximum score of 45. (Individual scores can be from 1 to 45) ...
4
votes
1answer
1k views

Which package to use for convergent and discriminant validity in R?

I am new to R and want to assess validity of some scales using R. Is there a package that I can use to test for convergent and discriminant validity in R?
2
votes
2answers
280 views

Removing human evaluator bias

I am working on machine translation evaluation, and am looking at ratings given by humans who are judging the quality of sentences produced by machine translation systems. The evaluators give fluency ...
8
votes
4answers
416 views

Why aren't all tests scored via item analysis/response theory?

Is there a statistical reason why item analysis/response theory isn't more widely applied? For instance, if a teacher gives a 25 question multiple choice test and finds that 10 questions were answered ...
1
vote
2answers
391 views

Rules of thumb for creating composite scales

Normally when I create an index I basically just take the or averages of the individual variables that comprise the index. But I'm wondering if one should / could not, take the product? E.g., a scale ...
5
votes
2answers
119 views

Sampling considerations in psychometric applications of item response theory

Background I have developed a recent interest in Item Response Theory and its applications. I am studying clinical psychology and am most interested in polytomous models aimed at modelling ...
14
votes
4answers
5k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
1
vote
3answers
950 views

Is there an R package which implements weighted maximum likelihood for Rasch models?

Is there a package in R which implements the weighted maximum likelihood method (Warm, 1996) for estimating the person parameters in Rasch Models?
5
votes
4answers
8k views

What are the primary differences between z-scores and t-scores, and are they both considered standard scores?

We are currently converting student test scores in this manner : ( ScaledScore - ScaledScore Mean ) / StdDeviation ) * 15 + 100 I was referring to this ...
2
votes
1answer
118 views

Assessing effect of intervention where aptitude tests used in pretest and posttest have some overlapping and some distinct items

In my research there is a pretest, and a treatment followed by a posttest. The pre and the posttests are similar but not exactly the same. A few questions were changed between the pre and posttests ...
5
votes
1answer
122 views

Testing the importance of an item among a finite set of items

Consider the following survey question: Q: How would you classify the importance for you of the following 5 items: A B C D E Assign to each item a number ...
11
votes
2answers
2k views

Assessing reliability of a questionnaire: dimensionality, problematic items, and whether to use alpha, lambda6 or some other index?

I am analyzing scores given by participants attending an experiment. I want to estimate the reliability of my questionnaire which is composed of 6 items aimed at estimating the attitude of the ...
10
votes
2answers
3k views

Recommended procedure for factor analysis on dichotomous data with R

I have to run a factor analysis on a dataset made up of dichotomous variables (0=yes, 1= no) and I don´t know if I'm on the right track. Using tetrachoric() I ...
5
votes
1answer
6k views

Intraclass correlation coefficient vs. F-test (one-way ANOVA)?

I'm a little confused regarding the intraclass correlation coefficient and one-way ANOVA. As I understand it, both tell you how similar observations within a group are, relative to observations in ...
7
votes
4answers
9k views

How to compute correlation between/within groups of variables?

I have a matrix of 1000 observations and 50 variables each measured on a 5-point scale. These variables are organized into groups, but there aren't an equal number of variables in each group. I'd ...
3
votes
0answers
2k views

How to carry out a Likert scale analysis?

Context: My research aims to assess whether parents who have been involved in a relationship with a social worker are at higher risk of child abuse/neglect. I am trying to establish whether there ...
21
votes
2answers
650 views

Detecting patterns of cheating on a multi-question exam

QUESTION: I have binary data on exam questions (correct/incorrect). Some individuals might have had prior access to a subset of questions and their correct answers. I don’t know who, how many, or ...
2
votes
2answers
359 views

What does it mean if there is a correlation between underlying factors in factor analysis?

I'm running an EFA using orthogonal/varimax rotation, and assigning variables to a factors based on maximum load (so only each variable only gets one factor). I then want to validate the model using ...
6
votes
3answers
1k views

Validating an existing questionnaire into another language

I am doing research and trying to use a questionnaire that is available in English in another language. Is there a systematic approach for validating a questionnaire in another language? What ...
3
votes
1answer
209 views

Is there a way to place a priori constraints on item difficulty under item response theory?

I have a dataset where the difficulty level of my items and skill of my participants is unknown, but I would like to be able to extract some metric of how individual participants do on the test. What ...
4
votes
2answers
454 views

When do you consider a variable is a latent variable?

The problem is to define when a variable might be considered as a latent variable. I am interested in how to describe a latent variable, and what are the properties of latent variables. My twofold ...
2
votes
1answer
2k views

Internal reliability for an ordinal scale

My teacher is asking whether it is possible to look at the Cronbachs's alpha when looking at the internal reliability of an ordinal scale. She thinks, because you use the mean of it, it is not ...
1
vote
2answers
184 views

Establishing that the population sampled of two separate surveys is the same

I have two surveys of two separate populations (I don't know that they are necessarily distinct, but they are from two different databases) that ask a similar set of questions. Some questions are ...
6
votes
2answers
268 views

Estimating the probability of a person getting a question right

We need to estimate the probability of a person getting a question right for a given content area given his history of getting questions in the same content area right in the past. We would also ...
8
votes
4answers
1k views

Can one validly reduce the numbers of items in a published Likert-scale?

[edits made in response to feedback- thanks :-) ] Doh! More edits! Sorry! Hello- I am doing some rather rough and ready data collection with a survey sent out to healthcare staff using a published ...
10
votes
3answers
3k views

Validating questionnaires

I am designing a questionnaire for my dissertation. I am in the process of validating the questionnaire I have applied a Cronbach's alpha test to the initial sample group. The responses to the ...
6
votes
2answers
1k views

Item Analysis for an R newbie

I am trying to assess a 20-item multliple choice test. I want to perform an item analysis such as can be found in this example. So for each question I want the P-value and the correlation with the ...
3
votes
1answer
1k views

Data transformation for Principal Components Analysis from different Likert scales

I have data from a survey comprised of several measures that used different Likert-type scaling (4-, 5-, and 6-point scales). I would like to run a principal components analysis using the data from ...
3
votes
2answers
339 views

The best measure of reliability for interval data between 0 and 1 [closed]

I have 6 sets of interval data each of which between 0 and 1. Each set, calculated by a computer program, is related to the degree of similarity between some sounds (pairwise). What do you think in ...
15
votes
2answers
10k views

Inter-rater reliability for ordinal or interval data

Which inter-rater reliability methods are most appropriate for ordinal or interval data? I believe that "Joint probability of agreement" or "Kappa" are designed for nominal data. Whilst "Pearson" and ...
3
votes
1answer
575 views

R package ltm: How to manipulate title on item response category characteristic curve plot

I have conducted an IRT analysis using package ltm in R 2.10. fit1 <- grm(data) When producing plots of item response characteristic curves (ICCs), ...
6
votes
3answers
3k views

Differences between tetrachoric and Pearson correlation

Currently I'm analysing around 300 items in the field of education. I'm interested in the dimensionality of the dataset. So I compute a matrix of tetrachoric correlation. The goal is to do a factor ...
2
votes
2answers
874 views

Confidence intervals on differences in choices in a GEE framework: methods and alternatives?

Say we have 5 items, and people are asked which item they like. Multiple answers are possible, but also no answer is possible. The people are categorized according to factors like gender, age, and so ...
3
votes
3answers
2k views

Classification after factor analysis

I have analysed several dimensions in a survey. Each part of the survey represents a theoretical dimension and is analysed with factorial analysis. I want to use scores from factor analysis to do a ...
10
votes
5answers
1k views

Is it appropriate to treat n-point Likert scale data as n trials from a binomial process?

I've never liked how people typically analyze data from Likert scales as if error were continuous & Gaussian when there are reasonable expectations that these assumptions are violated at least at ...
11
votes
3answers
4k views

Factor analysis of questionnaires composed of Likert items

I used to analyse items from a psychometric point of view. But now I am trying to analyse other types of questions on motivation and other topics. These questions are all on Likert scales. My initial ...
4
votes
3answers
319 views

Quantitative methods and statistics conferences in psychology?

What are the most significant annual conferences focusing on quantitative methods in psychology? This could include but is not limited to psychometrics, mathematical psychology, and statistical ...
5
votes
4answers
706 views

FA: Choosing Rotation matrix, based on “Simple Structure Criteria”

One of the most important issues in using factor analysis is its interpretation. Factor analysis often uses factor rotation to enhance its interpretation. After a satisfactory rotation, the rotated ...
7
votes
2answers
699 views

Reliability in Elicitation Exercise

I am trying to calculate the reliability in an elicitation exercise by analysing some test-retest questions given to the experts. The experts elicited a series of probability distributions which were ...