Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

4
votes
2answers
2k views

Can I combine several ordinal questions to create an index and/or composite measure?

I am using a survey that contains several questions about various dimensions of performance for policy research institutes. Here, performance in the policy arena is unpacked into things like: ...
0
votes
0answers
11 views

Interpreting summary of ICC results in R package “psych”

I have a dataframe "all.st.icc", where 27 subjects have measures of social behavior for each of 8 different years. First four rows of data are as follows: ...
0
votes
0answers
37 views

Is the use of PCA appropriate to validate a designed questionnaire?

I looked at the questions that may already have my answer but unfortunately it is not the case. I would like to ask again my question which has been revised to fit the rules. I am to measure 5 ...
0
votes
0answers
13 views

The differences between criterion validity and operational validity

In some psychological articles, I have seen the term "operational validity". It is often used instead of "criterion validity". What are the differences between these two concepts?
1
vote
1answer
20 views

Where I can find resources to learn how to calculate the sample size representativeness, and realiability and validity of questionnaires?

I'm totally newbie in psychometrics but starting from a research paper with full data I would like to understand: how to calculate if the sample size is representative; how to calculate reliability ...
1
vote
1answer
27 views

Questions regarding interpretation of item parameters: inconsistent results between classical item analysis and item response analysis

I was not trained as a psychometrician, so my questions may be concept-related. If you know different resources, books, or articles that can clear my confusions, please let me know. Thank you very ...
3
votes
1answer
162 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
0
votes
0answers
21 views

Attitude measurement – Score classification – Why using percentiles?

Once we have raw scores from a group of Likert scale items, why are percentiles used instead of considering all the possible scores from the total number items when classifying results of individuals ...
1
vote
1answer
455 views

Doing CFA on a known theoretical model, but having problems with convergent and discriminant validity

I did a survey based on the UTAUT model / theory with standard questions used. The N was 150. I am trying to validate the model for my survey with structural equation modeling (via the AMOS software). ...
1
vote
0answers
18 views

Combining different raters scores

In my experiment two raters evaluate 100 products, both expressing an "innovation score" on a likert scale (range is 1 to 5, where 1 is "not innovative" and 5 is "disruptive innovation"). I ...
4
votes
1answer
204 views

Omega H asymptotic

In the omega function in the psych package for R, we have several outputs including Hierarchical omega, Asymptotic omega and Total omega. I was reading about the outputs, but I find nothing about ...
2
votes
1answer
61 views

Item discrimination of poorly targeted items

In test theory, it appears to be widely recognized that very "easy" or "very difficult" items, relative to the ability of the sample of respondents, will have deflated correlations with the rest of ...
1
vote
1answer
214 views

Using Ltm package to calclate 3PL some lines of data seem to go missing

I am hoping someone can help with the following: I have been trying to use the ltm package in R to get person fit estimates using a 3PL model. The problem I have been having is that I start off ...
15
votes
4answers
7k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
1
vote
0answers
13 views

Which CFA model is the best fit in my translation & validation study?

I have translated and adapted a questionnaire (based on a 5 factor model; 23 items) into my native language by following ITC and MAPI guidelines. Forward / backward translation, expert panel review, ...
4
votes
1answer
92 views

Questions Regarding Item Response Models – the 3PL model vs. the mixture of 2PL and 3PL model

I currently work on an item analysis for an assessment. The assessment consists of 25 multiple-choice questions. Each question has 4 choices with one correct answer. I used different IRT models to ...
3
votes
1answer
41 views

Is it possible to do IRT distractor analysis using R?

I've been using the mirt package to analyse a few response matrixes I have through IRT. However, every time I need to perform distractor analysis, I end up having ...
1
vote
0answers
12 views

Correction of data using a correlation

Suppose I have measured the outcome variable A using a (psychophysical) test that determines the ability of a subject to discriminate between two stimuli with a certain difference (the variable X). It ...
1
vote
1answer
26 views

An invalid questionnare with valid resultant scores?

In what situation, a questionnaire could be invalid but its resultant scores could be valid? I received a comment from a journal article reviewer that says: In keeping with the preferred ...
1
vote
0answers
12 views

Is there any research that compares Likert scale questions to “satisfaction” scale questions?

I'm trying to put together a questionnaire for measuring employee satisfaction with an internal process. I'd like to figure out if there is reason to prefer: ...
3
votes
3answers
2k views

A multitrait-multimethod matrix and data set

I am working my way through Multitrait-Multimethod Matrix in a psychometrics class. We're only required to be able to analyze them but I'd really like to be able to construct them. I think I am able ...
0
votes
0answers
20 views

Is a representative sample necessary to assess construct validity?

To generalize results to a greater population, a representative sample is necessary. In developing a psychometrically valid instrument, via confirmatory factor analysis or item response theory, we are ...
6
votes
2answers
138 views

What is a good method to identify outliers in exam data?

I give my students an exam that has 8 questions on it. Each question is about a particular topic. The exam is made up on the fly by randomly selecting 1 question for each topic from a pool of ...
0
votes
0answers
35 views

How to build a perceptual scale using ratio measurements?

I have several physical samples of a material that vary according to some physical measure, e.g. hardness, roughness, brightness etc. Let's use brightness as the example. Now, I am not so much ...
16
votes
6answers
1k views

Introduction to structural equation modeling

I am asked by colleagues some help in this subject, that I don’t really know. They made hypotheses on the role of some latent variables in one study, and a referee asked them to formalize this in SEM. ...
3
votes
0answers
67 views

How high is too high with Cronbach's alpha?

Someone in my organization just sent me a scale they developed (modified items from five separate scales used in published research) to measure an employee's readiness for change. They tested the ...
0
votes
1answer
89 views

What is the statistic to calculate Cronbach's alpha for Polytomous items in R

Does anyone know, what is the function which can calculate Cronbach's alpha for polytomous items with different categories in R? Thank you
4
votes
1answer
234 views

How to apply published factor analysis loadings to sample data to compute factor scores?

I have data on 18 participants responding to a 30-item instrument. I also have a paper in which the researchers performed a principal axis factor analysis on the same survey where they report the item ...
2
votes
3answers
1k views

Is there an R package which implements weighted maximum likelihood for Rasch models?

Is there a package in R which implements the weighted maximum likelihood method (Warm, 1996) for estimating the person parameters in Rasch Models?
1
vote
1answer
28 views

Reading recommendations for psychometric refinement

a research project turned up a large number of survey instruments on a specific topic. There is a great deal of conceptual item overlap in these instruments, but they do differ in some key ways, ...
1
vote
0answers
28 views

Confidence interval for psychometric binomial GLM model on more than one subject in R

I'm trying to estimate the confidence interval of a psychometric curve (binomial probit GLM), for a population (now only two subjects). Suppose I've subject "a" and subject "b", which performs ...
3
votes
1answer
251 views

How to select next item in polytomous CAT / MIRT?

To make a Computerized Adaptive Test out of a sample of 20 dichotomous items a typical course of action would be to: 1) calibrate respondent data with a R package like mirt or ltm using Rasch, 2PL ...
0
votes
2answers
1k views

Error message from grm() in ltm: subscript out of bounds

I am analyzing a survey data set. One of the items in the survey is: ...
-1
votes
1answer
64 views

Sum of percentile rank scores of two or more variables

I need to sum some variables that are on different scales to make a new one. I'm wondering If it's correct to calculate the percentile scores first, and then to sum them up. Imagine I have to ...
7
votes
2answers
142 views

A person repeatedly selects the two most similar items out of three. How to model/estimate a perceptual distance between the items?

A person is given three items, say pictures of faces, and is asked to pick out which two of the three faces are the most similar. This is repeated a large number of times with different combinations ...
1
vote
0answers
25 views

Item Information Distorting Scale

I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...
4
votes
0answers
85 views

How to quickly identify participants responding randomly to self-report psychometric tests with many items?

Many psychological studies involve getting participants to answer a hundred or more closed ended questions. A standard context would be a personality test with 100 items where each item is answered on ...
2
votes
1answer
36 views

Publications in the social sciences or psychology based on freely available datasets?

There are a lot of datasets freely available across the web (see CV questions here, here, or here). Yet, I want the reverse: I am looking for a list of publications in the social sciences or ...
1
vote
0answers
25 views

Factor analysis without theoretical congruence

I've been working with certain personality theory, which I'm trying to measure in a sample of students. It's a foreign theory, so I created an instrument for my specific population. I got a Cronbach's ...
1
vote
0answers
34 views

Measure difference in answers on survey depending on item response options

For my thesis I sent out a questionnaire that has 10 questions. I set defaults: the first seven question had the default set to the desired answer and the last three questions had the default set to ...
2
votes
0answers
19 views

Estimating and using reliability metric with the same data

I'd like to know if it is reasonable to use the same set of data to estimate and use a reliability metric from repeated measures? I am mostly interesting by the minimal detectable change (MDC). But I ...
1
vote
1answer
52 views

Correlations within Factor in a psychometric measurement instrument

I am working with results of a psychometric test. There are twelve factors and each have three subfactors. My problem is that one of the subfactors correlates very highly 0.3 to 0.7 with subfactors of ...
2
votes
0answers
51 views

How to deal with floor effect

I am in the process of validating a five-items scale for measuring dependence on substance 'X'. I have collected data from 98 people who used the substance under consideration at least once weekly for ...
2
votes
0answers
45 views

Threshold for psychometric experiment with three (or more) possible choices

I'm running an experiment where subjects are viewing a stimulus, and they have to decide if the stimulus is brighter/darker/identical in respect to a standard image. Stimulus luminosity actually ...
0
votes
0answers
68 views

Estimates of regression coefficients at subject- vs group-level

For each of several subjects, I have mean reaction times for each of several stimuli. For each individual subject, I want to regress the RTs (as Y) against the stimuli (as X) and see if I get a slope ...
3
votes
0answers
205 views

What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
12
votes
2answers
797 views

What criteria must be met in order to conclude a 'ceiling effect' is occurring?

According to The SAGE Encyclopedia of Social Science Research Methods… [a] ceiling effect occurs when a measure possesses a distinct upper limit for potential responses and a large concentration ...
0
votes
0answers
39 views

Testing face validity of a questionnaire

I would like to validate a questionnaire for my population, using a "think-aloud" (protocol analysis) method. The study looks at IT user satisfaction. Do you think my participants have to use the ...
0
votes
0answers
69 views

Principal component analysis, psychometrics, and number of items required

I saw that questions regarding sample sizes and PCA are asked here quite frequently. However, I was not able to find exactly the information I need. I plan to do a psychometric study and conduct a ...