Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

0
votes
2answers
825 views

Error message from grm() in ltm: subscript out of bounds

I am analyzing a survey data set. One of the items in the survey is: ...
1
vote
0answers
19 views

Measure difference in answers on survey depending on item response options

For my thesis I sent out a questionnaire that has 10 questions. I set defaults: the first seven question had the default set to the desired answer and the last three questions had the default set to ...
2
votes
0answers
16 views

Estimating and using reliability metric with the same data

I'd like to know if it is reasonable to use the same set of data to estimate and use a reliability metric from repeated measures? I am mostly interesting by the minimal detectable change (MDC). But I ...
1
vote
1answer
33 views

Correlations within Factor in a psychometric measurement instrument

I am working with results of a psychometric test. There are twelve factors and each have three subfactors. My problem is that one of the subfactors correlates very highly 0.3 to 0.7 with subfactors of ...
2
votes
0answers
32 views

How to deal with floor effect

I am in the process of validating a five-items scale for measuring dependence on substance 'X'. I have collected data from 98 people who used the substance under consideration at least once weekly for ...
1
vote
0answers
31 views

Threshold for psychometric experiment with three (or more) possible choices

I'm running an experiment where subjects are viewing a stimulus, and they have to decide if the stimulus is brighter/darker/identical in respect to a standard image. Stimulus luminosity actually ...
3
votes
2answers
658 views

Can I combine several ordinal questions to create an index and/or composite measure?

I am using a survey that contains several questions about various dimensions of performance for policy research institutes. Here, performance in the policy arena is unpacked into things like: ...
1
vote
1answer
84 views

Using Ltm package to calclate 3PL some lines of data seem to go missing

I am hoping someone can help with the following: I have been trying to use the ltm package in R to get person fit estimates using a 3PL model. The problem I have been having is that I start off ...
3
votes
1answer
125 views

Omega H asymptotic

In the omega function in the psych package for R, we have several outputs including Hierarchical omega, Asymptotic omega and Total omega. I was reading about the outputs, but I find nothing about ...
3
votes
1answer
92 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
0
votes
0answers
25 views

Estimates of regression coefficients at subject- vs group-level

For each of several subjects, I have mean reaction times for each of several stimuli. For each individual subject, I want to regress the RTs (as Y) against the stimuli (as X) and see if I get a slope ...
3
votes
0answers
66 views

What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
11
votes
2answers
263 views

What criteria must be met in order to conclude a 'ceiling effect' is occurring?

According to The SAGE Encyclopedia of Social Science Research Methods… [a] ceiling effect occurs when a measure possesses a distinct upper limit for potential responses and a large concentration ...
0
votes
0answers
17 views

Testing face validity of a questionnaire

I would like to validate a questionnaire for my population, using a "think-aloud" (protocol analysis) method. The study looks at IT user satisfaction. Do you think my participants have to use the ...
0
votes
0answers
30 views

Principal component analysis, psychometrics, and number of items required

I saw that questions regarding sample sizes and PCA are asked here quite frequently. However, I was not able to find exactly the information I need. I plan to do a psychometric study and conduct a ...
0
votes
1answer
43 views

Eliminated items in Factor Analysis

I'd like to get your opinions on how to interpret items that had to be eliminated in factor analysis (FA). I've been researching consumer shopping motivations and ran a survey with 40 items, which ...
6
votes
3answers
271 views

What is the reasoning behind the formulae for the different standard errors of measurement?

McManus (2012) discusses the three standard errors of measurement and provides the following definitions. $SE_{meas} = SD \sqrt{(1 - reliability)}$ The standard error of measurement is an estimate ...
3
votes
4answers
210 views

How can there be a standard deviation of true scores?

Dudek (1979) describes the following formula as "the standard deviation of true scores when the observed score is held constant" (p. 337): $SE_{est} = SD \sqrt{(reliability Ă— (1 - reliability))}$ ...
1
vote
0answers
13 views

Is there an extension of multinomial process models that can model several dependent variables?

In the setting of psychological experiments, where a categorical response has to be given, the same response can, in theory, be generated by different latent processes. For example, in an experiment ...
7
votes
2answers
467 views

How is the Spearman-Brown prophecy formula affected by questions of differing difficulties?

How are the results of the Spearman-Brown prophecy formula affected by having test questions of differing difficulties or raters who are easy or hard graders. One respected text says the S-B is ...
6
votes
3answers
117 views

How to correctly interpret a parallel analysis in exploratory factor analysis?

Some scientific papers report results of parallel analysis of principal axis factor analysis in a way inconsistent with my understanding of the methodology. What am I missing? Am I wrong or are they. ...
3
votes
1answer
57 views

Why change items which don't change Cronbach Alpha?

I am doing peer review on a paper submission for a journal, and I am somewhat out of my depth. The authors are developing a psychometric scale. They did a pilot study with 45 items. They measured ...
11
votes
3answers
4k views

Factor analysis of questionnaires composed of Likert items

I used to analyse items from a psychometric point of view. But now I am trying to analyse other types of questions on motivation and other topics. These questions are all on Likert scales. My initial ...
0
votes
2answers
83 views

Neutral option on Likert Scale

I would like to know how do we analyze or regard the Neutral response (such as Don't Know) in a questionnaire using the following Likert Scale: 1:Strongly Disagree 2:Disagree 3:Don't Know 4:Agree ...
7
votes
4answers
2k views

Is 0 a valid value in a Likert scale?

I have carried out my pilot study on language learning motivation using a 6 point Likert scale but from 0 (strongly disagree) to 5 (very much agree). I noticed a colleague in his survey used 1 to 6. ...
0
votes
0answers
42 views

Correlation significance test with standardized z scores

I am using spss I am looking at a criterion-validity concurrent design. I had to transform two groups performance ratings into z scores because there were differences in how raters rated them on the ...
1
vote
0answers
43 views

Map a normal distribution N(x,s) to an ordinal response variable

I am working on a simulation. I am going to extract a series of normally distributed values from a distribution $N(x,s)$ whose mean $x$ and variance $s$ is known. I want to pretend that such ...
0
votes
0answers
14 views

theoretical profile correlation

My data (n > 4000) contains the results of two separate tests (instruments). I have a profile made from likert scales for one of the instruments (S1, S2, ... S20). I want to find the corresponding ...
3
votes
0answers
49 views

How can IRT-Models be understood in GLM/ SEM Framework? (Predict Learning with added Paradata-Covariates)

I'll be working with data from an intelligent tutor system similar to one studied in the KDD-Cup 2010 on student performance prediction and plan to use IRT models to infer item and ability parameters. ...
6
votes
1answer
91 views

Item reliability or fit in IRT-based adaptive tests?

My background is in machine learning and statistics, but I am relatively new to psychometrics and testing. Nearly all of the literature I've found on item reliability refers to Chronbach's alpha or ...
5
votes
3answers
5k views

Is it acceptable to use Cronbach's alpha to assess reliability of questionnaire composed of categorical and conditional items?

Background: I have data on a questionnaire made up of categorical factual items. Some of them are binary and others have more than two categories. For example: ...
2
votes
2answers
2k views

Omega vs. alpha reliability

I wonder if someone can explain what is the main difference between omega and alpha reliabilities? I understand an omega reliability is based on hierarchical factor model as shown in the following ...
0
votes
0answers
50 views

How to test measurement invariance with PLS?

I have two questions about structural equations modeling: To perform comparisons of model estimates for different groups (3 clusters), do I always have to analize measurement invariance first? If ...
2
votes
3answers
197 views

Can statistics really be used in education to determine test scores?

I am a middle school teacher. My district prides itself on having the kids take 3 tests during the year to predict their score, and my teaching, on "the real test" at the end of the year. My question: ...
10
votes
2answers
228 views

Identification of useless questions from a questionnaire

I'm developing a questionnaire. To improve its reliability and validity I want to use statistical methods. I want to eliminate questions whose answers are always the same. This means that nearly all ...
1
vote
1answer
8k views

How to calculate validity of questionnaire in SPSS?

I have 3 types of questionnaires, A 15 item scale with each item on a 7-point Likert type scale. Six subscales each containing different number of items 21 subscales each containing 4 items. ...
0
votes
2answers
56 views

Can I turn a latent variable be treated as an observed variable?

I am a doctoral candidate and for my dissertation research I am using two standardized scales as independent variables. Because these have not been used with my particular population, I needed to do ...
4
votes
3answers
241 views

Are discrimination parameters in two-parameter IRT models only specific to items?

Discrimination parameters in two-parameter model from IRT are usually considered item parameters. But I've come to doubt it. Think about psychophysics; for example, detecting luminance. I don't ...
14
votes
4answers
5k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
11
votes
3answers
3k views

Validating questionnaires

I am designing a questionnaire for my dissertation. I am in the process of validating the questionnaire I have applied a Cronbach's alpha test to the initial sample group. The responses to the ...
4
votes
4answers
303 views

If a measure is valid (but not necesarily reliable), can it be consistently replicated?

I had this question on an exam, and I was positive that the answer was A. The Beck Depression Inventory is a scale intended to measure depression levels, with higher scores indicative of higher ...
5
votes
1answer
749 views
5
votes
1answer
506 views

Good internal factor structure but poor cronbach alpha?

I am running a CFA and getting good fit indices (CFI = .99, RMSEA = .01) for a uni-dimensional scale. However, when I test for internal consistency, I get poor Cronbach alphas (alpha = .6). I have ...
0
votes
1answer
76 views

Several items for one construct

I am a dummy in statistics but need to accomplish some simple survey analyses. My question is: if I have several questions (items) for one construct (say, questions JS1-JS3 for job satisfaction) with ...
2
votes
2answers
2k views

Likert scale and yes/no calculation of one score

My survey is based on observations and interviews. The observation questions are yes/no type questions and interview questions are 4 points. $1=$ never, $2=$ sometimes, $3=$ usually, and $4=$ ...
0
votes
1answer
118 views

Calculating total score for a scale including polytomous and dichotomous items [duplicate]

I have a variable that is measured using a 13-item, 4-point Likert scale with the exception of one item (yes $=1$, no $=2)$. How can I calculate the total score for this variable?
1
vote
3answers
922 views

Is there an R package which implements weighted maximum likelihood for Rasch models?

Is there a package in R which implements the weighted maximum likelihood method (Warm, 1996) for estimating the person parameters in Rasch Models?
1
vote
0answers
222 views

Intra- and inter- observer error, and the intra-class correlation coefficient (ICC)

I have 3 observers, who each perform multiple observations in which they count a phenomenon. I'm trying to quantify both inter- and intra- observer error for this study. I'm new to this and am finding ...
10
votes
5answers
1k views

Is it appropriate to treat n-point Likert scale data as n trials from a binomial process?

I've never liked how people typically analyze data from Likert scales as if error were continuous & Gaussian when there are reasonable expectations that these assumptions are violated at least at ...
3
votes
3answers
2k views

A multitrait-multimethod matrix and data set

I am working my way through Multitrait-Multimethod Matrix in a psychometrics class. We're only required to be able to analyze them but I'd really like to be able to construct them. I think I am able ...