Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

-1
votes
1answer
30 views

Sum of percentile rank scores of two or more variables

I need to sum some variables that are on different scales to make a new one. I'm wondering If it's correct to calculate the percentile scores first, and then to sum them up. Imagine I have to ...
1
vote
0answers
19 views

Item Information Distorting Scale

I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...
7
votes
2answers
128 views

A person repeatedly selects the two most similar items out of three. How to model/estimate a perceptual distance between the items?

A person is given three items, say pictures of faces, and is asked to pick out which two of the three faces are the most similar. This is repeated a large number of times with different combinations ...
4
votes
0answers
62 views

How to quickly identify participants responding randomly to self-report psychometric tests with many items?

Many psychological studies involve getting participants to answer a hundred or more closed ended questions. A standard context would be a personality test with 100 items where each item is answered on ...
1
vote
1answer
34 views

Publications in the social sciences or psychology based on freely available datasets?

There are a lot of datasets freely available across the web (see CV questions here, here, or here). Yet, I want the reverse: I am looking for a list of publications in the social sciences or ...
1
vote
0answers
21 views

Factor analysis without theoretical congruence

I've been working with certain personality theory, which I'm trying to measure in a sample of students. It's a foreign theory, so I created an instrument for my specific population. I got a Cronbach's ...
0
votes
0answers
25 views

Identifying the number of conditions by which variance components will be divided in Generalizability Theory using QME R package

I'm using QME R Package (can be downloaded from here) to calculate Generalizability ...
1
vote
0answers
27 views

Measure difference in answers on survey depending on item response options

For my thesis I sent out a questionnaire that has 10 questions. I set defaults: the first seven question had the default set to the desired answer and the last three questions had the default set to ...
2
votes
0answers
19 views

Estimating and using reliability metric with the same data

I'd like to know if it is reasonable to use the same set of data to estimate and use a reliability metric from repeated measures? I am mostly interesting by the minimal detectable change (MDC). But I ...
2
votes
0answers
40 views

Threshold for psychometric experiment with three (or more) possible choices

I'm running an experiment where subjects are viewing a stimulus, and they have to decide if the stimulus is brighter/darker/identical in respect to a standard image. Stimulus luminosity actually ...
0
votes
0answers
42 views

Estimates of regression coefficients at subject- vs group-level

For each of several subjects, I have mean reaction times for each of several stimuli. For each individual subject, I want to regress the RTs (as Y) against the stimuli (as X) and see if I get a slope ...
2
votes
0answers
45 views

How to deal with floor effect

I am in the process of validating a five-items scale for measuring dependence on substance 'X'. I have collected data from 98 people who used the substance under consideration at least once weekly for ...
1
vote
1answer
42 views

Correlations within Factor in a psychometric measurement instrument

I am working with results of a psychometric test. There are twelve factors and each have three subfactors. My problem is that one of the subfactors correlates very highly 0.3 to 0.7 with subfactors of ...
0
votes
0answers
30 views

Testing face validity of a questionnaire

I would like to validate a questionnaire for my population, using a "think-aloud" (protocol analysis) method. The study looks at IT user satisfaction. Do you think my participants have to use the ...
0
votes
0answers
52 views

Principal component analysis, psychometrics, and number of items required

I saw that questions regarding sample sizes and PCA are asked here quite frequently. However, I was not able to find exactly the information I need. I plan to do a psychometric study and conduct a ...
0
votes
1answer
67 views

Eliminated items in Factor Analysis

I'd like to get your opinions on how to interpret items that had to be eliminated in factor analysis (FA). I've been researching consumer shopping motivations and ran a survey with 40 items, which ...
3
votes
0answers
118 views

What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
1
vote
0answers
13 views

Is there an extension of multinomial process models that can model several dependent variables?

In the setting of psychological experiments, where a categorical response has to be given, the same response can, in theory, be generated by different latent processes. For example, in an experiment ...
6
votes
3answers
329 views

How to correctly interpret a parallel analysis in exploratory factor analysis?

Some scientific papers report results of parallel analysis of principal axis factor analysis in a way inconsistent with my understanding of the methodology. What am I missing? Am I wrong or are they. ...
3
votes
1answer
84 views

Why change items which don't change Cronbach Alpha?

I am doing peer review on a paper submission for a journal, and I am somewhat out of my depth. The authors are developing a psychometric scale. They did a pilot study with 45 items. They measured ...
0
votes
2answers
217 views

Neutral option on Likert Scale

I would like to know how do we analyze or regard the Neutral response (such as Don't Know) in a questionnaire using the following Likert Scale: 1:Strongly Disagree 2:Disagree 3:Don't Know 4:Agree ...
7
votes
2answers
1k views

How is the Spearman-Brown prophecy formula affected by questions of differing difficulties?

How are the results of the Spearman-Brown prophecy formula affected by having test questions of differing difficulties or raters who are easy or hard graders. One respected text says the S-B is ...
12
votes
2answers
410 views

What criteria must be met in order to conclude a 'ceiling effect' is occurring?

According to The SAGE Encyclopedia of Social Science Research Methods… [a] ceiling effect occurs when a measure possesses a distinct upper limit for potential responses and a large concentration ...
0
votes
0answers
49 views

Correlation significance test with standardized z scores

I am using spss I am looking at a criterion-validity concurrent design. I had to transform two groups performance ratings into z scores because there were differences in how raters rated them on the ...
0
votes
0answers
17 views

theoretical profile correlation

My data (n > 4000) contains the results of two separate tests (instruments). I have a profile made from likert scales for one of the instruments (S1, S2, ... S20). I want to find the corresponding ...
3
votes
0answers
65 views

How can IRT-Models be understood in GLM/ SEM Framework? (Predict Learning with added Paradata-Covariates)

I'll be working with data from an intelligent tutor system similar to one studied in the KDD-Cup 2010 on student performance prediction and plan to use IRT models to infer item and ability parameters. ...
6
votes
1answer
121 views

Item reliability or fit in IRT-based adaptive tests?

My background is in machine learning and statistics, but I am relatively new to psychometrics and testing. Nearly all of the literature I've found on item reliability refers to Chronbach's alpha or ...
0
votes
0answers
71 views

How to test measurement invariance with PLS?

I have two questions about structural equations modeling: To perform comparisons of model estimates for different groups (3 clusters), do I always have to analize measurement invariance first? If ...
2
votes
3answers
213 views

Can statistics really be used in education to determine test scores?

I am a middle school teacher. My district prides itself on having the kids take 3 tests during the year to predict their score, and my teaching, on "the real test" at the end of the year. My question: ...
10
votes
2answers
272 views

Identification of useless questions from a questionnaire

I'm developing a questionnaire. To improve its reliability and validity I want to use statistical methods. I want to eliminate questions whose answers are always the same. This means that nearly all ...
4
votes
4answers
525 views

If a measure is valid (but not necesarily reliable), can it be consistently replicated?

I had this question on an exam, and I was positive that the answer was A. The Beck Depression Inventory is a scale intended to measure depression levels, with higher scores indicative of higher ...
0
votes
1answer
102 views

Several items for one construct

I am a dummy in statistics but need to accomplish some simple survey analyses. My question is: if I have several questions (items) for one construct (say, questions JS1-JS3 for job satisfaction) with ...
3
votes
1answer
124 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
0
votes
1answer
151 views

Calculating total score for a scale including polytomous and dichotomous items [duplicate]

I have a variable that is measured using a 13-item, 4-point Likert scale with the exception of one item (yes $=1$, no $=2)$. How can I calculate the total score for this variable?
0
votes
2answers
64 views

Can I turn a latent variable be treated as an observed variable?

I am a doctoral candidate and for my dissertation research I am using two standardized scales as independent variables. Because these have not been used with my particular population, I needed to do ...
1
vote
1answer
130 views

Using Ltm package to calclate 3PL some lines of data seem to go missing

I am hoping someone can help with the following: I have been trying to use the ltm package in R to get person fit estimates using a 3PL model. The problem I have been having is that I start off ...
1
vote
0answers
308 views

Intra- and inter- observer error, and the intra-class correlation coefficient (ICC)

I have 3 observers, who each perform multiple observations in which they count a phenomenon. I'm trying to quantify both inter- and intra- observer error for this study. I'm new to this and am finding ...
5
votes
1answer
741 views

Good internal factor structure but poor cronbach alpha?

I am running a CFA and getting good fit indices (CFI = .99, RMSEA = .01) for a uni-dimensional scale. However, when I test for internal consistency, I get poor Cronbach alphas (alpha = .6). I have ...
1
vote
0answers
43 views

Map a normal distribution N(x,s) to an ordinal response variable

I am working on a simulation. I am going to extract a series of normally distributed values from a distribution $N(x,s)$ whose mean $x$ and variance $s$ is known. I want to pretend that such ...
4
votes
3answers
205 views

How to check psychometric type of scales?

It is well known, that there are diffrent types of scales (see Wikipedia Level of measurement). In psycholinguistics there is Semantic Differential technique. It uses a scale (usually 5 or 7 grades) ...
1
vote
1answer
403 views

Cronbach's alpha vs. KR20/KR21 in reliability studies

Should we calculate Cronbach's alpha instead of KR20/KR21 for measuring the reliability of an achievement test, or a test with dichotomous data?
10
votes
2answers
3k views

Where do the descriptors for Cronbach's alpha values come from (e.g., poor, excellent)?

It seems fairly common to describe Cronbach's alpha values as follows: α ≥ 0.9 Excellent 0.7 ≤ α < 0.9 Good 0.6 ≤ α < 0.7 Acceptable 0.5 ≤ α < 0.6 Poor α < 0.5 Unacceptable Where ...
3
votes
1answer
383 views

What is the correct use of Cronbach's alpha?

I am trying to use Cronbach's alpha in order to see if some variables are measuring the "same", but I have some doubts about the correct use: Can I compute alpha for numerical, ordinal or ...
1
vote
0answers
121 views

Confirmatory factor analysis for a newly constructed questionnaire

I am about to develop a new questionnaire with 142 questions I have performed exploratory factor analysis and 30 latent factors were extracted. I was wondering if it is necessary to perform ...
1
vote
0answers
338 views

Doing CFA on a known theoretical model, but having problems with convergent and discriminant validity

I did a survey based on the UTAUT model / theory with standard questions used. The N was 150. I am trying to validate the model for my survey with structural equation modeling (via the AMOS software). ...
3
votes
1answer
210 views

Item response theory vs. summative Likert scale

If the items to be "summed" or combined to create an overall index are collectively the underlying construct (i.e. I am trying to measure compliance to an intervention which has different ...
1
vote
1answer
129 views

Two treatments - How do I compute reliability estimates like Cronbach $\alpha$?

I randomized my survey, and now I have my data on SPSS (and all cleaned up) but when trying to run my Cronbach's alpha I have a problem. Because I have two treatments (sexualised and normal) all my ...
3
votes
0answers
108 views

Should I run measurement invariance if confirmatory factor analysis is not affirmative?

I have a repeated measures design for a Likert scale with 10 items. This scale is assumed to be unidimensional and this is one of the research questions along with the scales measurement invariance ...
0
votes
2answers
145 views

Should I use raw scores or profile scores for variables in a regression model?

I'm trying to build a regression model to predict subjective memory reports based on the following predictors: objective memory test, illness perceptions, verbal fluency, and mood. I also want to ...
1
vote
1answer
310 views

Using existing questionnaire

I will be doing my colloquium and will be using existing questionnaires. I would like to know if there are authors/authorities who say that an existing questionnaire need not be validated. The ...