Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

1
vote
1answer
132 views

Using Ltm package to calclate 3PL some lines of data seem to go missing

I am hoping someone can help with the following: I have been trying to use the ltm package in R to get person fit estimates using a 3PL model. The problem I have been having is that I start off ...
0
votes
2answers
1k views

Error message from grm() in ltm: subscript out of bounds

I am analyzing a survey data set. One of the items in the survey is: ...
4
votes
1answer
165 views

Omega H asymptotic

In the omega function in the psych package for R, we have several outputs including Hierarchical omega, Asymptotic omega and Total omega. I was reading about the outputs, but I find nothing about ...
-1
votes
1answer
31 views

Sum of percentile rank scores of two or more variables

I need to sum some variables that are on different scales to make a new one. I'm wondering If it's correct to calculate the percentile scores first, and then to sum them up. Imagine I have to ...
7
votes
2answers
128 views

A person repeatedly selects the two most similar items out of three. How to model/estimate a perceptual distance between the items?

A person is given three items, say pictures of faces, and is asked to pick out which two of the three faces are the most similar. This is repeated a large number of times with different combinations ...
1
vote
0answers
20 views

Item Information Distorting Scale

I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...
3
votes
1answer
124 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
4
votes
0answers
62 views

How to quickly identify participants responding randomly to self-report psychometric tests with many items?

Many psychological studies involve getting participants to answer a hundred or more closed ended questions. A standard context would be a personality test with 100 items where each item is answered on ...
1
vote
1answer
199 views

How to select next item in polytomous CAT / MIRT?

To make a Computerized Adaptive Test out of a sample of 20 dichotomous items a typical course of action would be to: 1) calibrate respondent data with a R package like mirt or ltm using Rasch, 2PL ...
1
vote
1answer
34 views

Publications in the social sciences or psychology based on freely available datasets?

There are a lot of datasets freely available across the web (see CV questions here, here, or here). Yet, I want the reverse: I am looking for a list of publications in the social sciences or ...
1
vote
0answers
21 views

Factor analysis without theoretical congruence

I've been working with certain personality theory, which I'm trying to measure in a sample of students. It's a foreign theory, so I created an instrument for my specific population. I got a Cronbach's ...
1
vote
0answers
27 views

Measure difference in answers on survey depending on item response options

For my thesis I sent out a questionnaire that has 10 questions. I set defaults: the first seven question had the default set to the desired answer and the last three questions had the default set to ...
2
votes
0answers
19 views

Estimating and using reliability metric with the same data

I'd like to know if it is reasonable to use the same set of data to estimate and use a reliability metric from repeated measures? I am mostly interesting by the minimal detectable change (MDC). But I ...
1
vote
1answer
42 views

Correlations within Factor in a psychometric measurement instrument

I am working with results of a psychometric test. There are twelve factors and each have three subfactors. My problem is that one of the subfactors correlates very highly 0.3 to 0.7 with subfactors of ...
2
votes
0answers
45 views

How to deal with floor effect

I am in the process of validating a five-items scale for measuring dependence on substance 'X'. I have collected data from 98 people who used the substance under consideration at least once weekly for ...
2
votes
0answers
40 views

Threshold for psychometric experiment with three (or more) possible choices

I'm running an experiment where subjects are viewing a stimulus, and they have to decide if the stimulus is brighter/darker/identical in respect to a standard image. Stimulus luminosity actually ...
3
votes
2answers
1k views

Can I combine several ordinal questions to create an index and/or composite measure?

I am using a survey that contains several questions about various dimensions of performance for policy research institutes. Here, performance in the policy arena is unpacked into things like: ...
0
votes
0answers
42 views

Estimates of regression coefficients at subject- vs group-level

For each of several subjects, I have mean reaction times for each of several stimuli. For each individual subject, I want to regress the RTs (as Y) against the stimuli (as X) and see if I get a slope ...
3
votes
0answers
118 views

What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
12
votes
2answers
419 views

What criteria must be met in order to conclude a 'ceiling effect' is occurring?

According to The SAGE Encyclopedia of Social Science Research Methods… [a] ceiling effect occurs when a measure possesses a distinct upper limit for potential responses and a large concentration ...
0
votes
0answers
30 views

Testing face validity of a questionnaire

I would like to validate a questionnaire for my population, using a "think-aloud" (protocol analysis) method. The study looks at IT user satisfaction. Do you think my participants have to use the ...
0
votes
0answers
53 views

Principal component analysis, psychometrics, and number of items required

I saw that questions regarding sample sizes and PCA are asked here quite frequently. However, I was not able to find exactly the information I need. I plan to do a psychometric study and conduct a ...
0
votes
1answer
67 views

Eliminated items in Factor Analysis

I'd like to get your opinions on how to interpret items that had to be eliminated in factor analysis (FA). I've been researching consumer shopping motivations and ran a survey with 40 items, which ...
6
votes
3answers
281 views

What is the reasoning behind the formulae for the different standard errors of measurement?

McManus (2012) discusses the three standard errors of measurement and provides the following definitions. $SE_{meas} = SD \sqrt{(1 - reliability)}$ The standard error of measurement is an estimate ...
3
votes
4answers
241 views

How can there be a standard deviation of true scores?

Dudek (1979) describes the following formula as "the standard deviation of true scores when the observed score is held constant" (p. 337): $SE_{est} = SD \sqrt{(reliability Ă— (1 - reliability))}$ ...
1
vote
0answers
13 views

Is there an extension of multinomial process models that can model several dependent variables?

In the setting of psychological experiments, where a categorical response has to be given, the same response can, in theory, be generated by different latent processes. For example, in an experiment ...
7
votes
2answers
1k views

How is the Spearman-Brown prophecy formula affected by questions of differing difficulties?

How are the results of the Spearman-Brown prophecy formula affected by having test questions of differing difficulties or raters who are easy or hard graders. One respected text says the S-B is ...
6
votes
3answers
339 views

How to correctly interpret a parallel analysis in exploratory factor analysis?

Some scientific papers report results of parallel analysis of principal axis factor analysis in a way inconsistent with my understanding of the methodology. What am I missing? Am I wrong or are they. ...
3
votes
1answer
85 views

Why change items which don't change Cronbach Alpha?

I am doing peer review on a paper submission for a journal, and I am somewhat out of my depth. The authors are developing a psychometric scale. They did a pilot study with 45 items. They measured ...
11
votes
3answers
5k views

Factor analysis of questionnaires composed of Likert items

I used to analyse items from a psychometric point of view. But now I am trying to analyse other types of questions on motivation and other topics. These questions are all on Likert scales. My initial ...
0
votes
2answers
219 views

Neutral option on Likert Scale

I would like to know how do we analyze or regard the Neutral response (such as Don't Know) in a questionnaire using the following Likert Scale: 1:Strongly Disagree 2:Disagree 3:Don't Know 4:Agree ...
7
votes
4answers
3k views

Is 0 a valid value in a Likert scale?

I have carried out my pilot study on language learning motivation using a 6 point Likert scale but from 0 (strongly disagree) to 5 (very much agree). I noticed a colleague in his survey used 1 to 6. ...
0
votes
0answers
49 views

Correlation significance test with standardized z scores

I am using spss I am looking at a criterion-validity concurrent design. I had to transform two groups performance ratings into z scores because there were differences in how raters rated them on the ...
1
vote
0answers
43 views

Map a normal distribution N(x,s) to an ordinal response variable

I am working on a simulation. I am going to extract a series of normally distributed values from a distribution $N(x,s)$ whose mean $x$ and variance $s$ is known. I want to pretend that such ...
0
votes
0answers
17 views

theoretical profile correlation

My data (n > 4000) contains the results of two separate tests (instruments). I have a profile made from likert scales for one of the instruments (S1, S2, ... S20). I want to find the corresponding ...
3
votes
0answers
66 views

How can IRT-Models be understood in GLM/ SEM Framework? (Predict Learning with added Paradata-Covariates)

I'll be working with data from an intelligent tutor system similar to one studied in the KDD-Cup 2010 on student performance prediction and plan to use IRT models to infer item and ability parameters. ...
6
votes
1answer
121 views

Item reliability or fit in IRT-based adaptive tests?

My background is in machine learning and statistics, but I am relatively new to psychometrics and testing. Nearly all of the literature I've found on item reliability refers to Chronbach's alpha or ...
5
votes
3answers
6k views

Is it acceptable to use Cronbach's alpha to assess reliability of questionnaire composed of categorical and conditional items?

Background: I have data on a questionnaire made up of categorical factual items. Some of them are binary and others have more than two categories. For example: ...
2
votes
2answers
2k views

Omega vs. alpha reliability

I wonder if someone can explain what is the main difference between omega and alpha reliabilities? I understand an omega reliability is based on hierarchical factor model as shown in the following ...
0
votes
0answers
72 views

How to test measurement invariance with PLS?

I have two questions about structural equations modeling: To perform comparisons of model estimates for different groups (3 clusters), do I always have to analize measurement invariance first? If ...
2
votes
3answers
213 views

Can statistics really be used in education to determine test scores?

I am a middle school teacher. My district prides itself on having the kids take 3 tests during the year to predict their score, and my teaching, on "the real test" at the end of the year. My question: ...
10
votes
2answers
273 views

Identification of useless questions from a questionnaire

I'm developing a questionnaire. To improve its reliability and validity I want to use statistical methods. I want to eliminate questions whose answers are always the same. This means that nearly all ...
1
vote
1answer
11k views

How to calculate validity of questionnaire in SPSS?

I have 3 types of questionnaires, A 15 item scale with each item on a 7-point Likert type scale. Six subscales each containing different number of items 21 subscales each containing 4 items. ...
0
votes
2answers
64 views

Can I turn a latent variable be treated as an observed variable?

I am a doctoral candidate and for my dissertation research I am using two standardized scales as independent variables. Because these have not been used with my particular population, I needed to do ...
4
votes
3answers
265 views

Are discrimination parameters in two-parameter IRT models only specific to items?

Discrimination parameters in two-parameter model from IRT are usually considered item parameters. But I've come to doubt it. Think about psychophysics; for example, detecting luminance. I don't ...
14
votes
4answers
6k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
10
votes
3answers
4k views

Validating questionnaires

I am designing a questionnaire for my dissertation. I am in the process of validating the questionnaire I have applied a Cronbach's alpha test to the initial sample group. The responses to the ...
4
votes
4answers
532 views

If a measure is valid (but not necesarily reliable), can it be consistently replicated?

I had this question on an exam, and I was positive that the answer was A. The Beck Depression Inventory is a scale intended to measure depression levels, with higher scores indicative of higher ...