Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

5
votes
1answer
28 views

Item reliability or fit in IRT-based adaptive tests?

My background is in machine learning and statistics, but I am relatively new to psychometrics and testing. Nearly all of the literature I've found on item reliability refers to Chronbach's alpha or ...
0
votes
0answers
10 views

How to can IRT-Models be understood in GLM/ SEM Framework? (Predict Learning with added Paradata-Covariates)

I'll be working with data from an a Intelligent Tutor System similar to one studied in the KDD-Cup 2010 on Student Performance Prediction and plan to use IRT-Models to infer item and ability ...
5
votes
3answers
4k views

Is it acceptable to use Cronbach's alpha to assess reliability of questionnaire composed of categorical and conditional items?

Background: I have data on a questionnaire made up of categorical factual items. Some of them are binary and others have more than two categories. For example: ...
2
votes
2answers
1k views

Omega vs. alpha reliability

I wonder if someone can explain what is the main difference between omega and alpha reliabilities? I understand an omega reliability is based on hierarchical factor model as shown in the following ...
0
votes
2answers
609 views

Error message from grm() in ltm: subscript out of bounds

I am analyzing a survey data set. One of the items in the survey is: ...
0
votes
0answers
11 views

How to test measurement invariance with PLS?

I have two questions about structural equations modeling: To perform comparisons of model estimates for different groups (3 clusters), do I always have to analize measurement invariance first? If ...
2
votes
1answer
51 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
1
vote
1answer
58 views

Using Ltm package to calclate 3PL some lines of data seem to go missing

I am hoping someone can help with the following: I have been trying to use the ltm package in R to get person fit estimates using a 3PL model. The problem I have been having is that I start off ...
2
votes
3answers
172 views

Can statistics really be used in education to determine test scores?

I am a middle school teacher. My district prides itself on having the kids take 3 tests during the year to predict their score, and my teaching, on "the real test" at the end of the year. My question: ...
3
votes
1answer
95 views

Omega H asymptotic

In the omega function in the psych package for R, we have several outputs including Hierarchical omega, Asymptotic omega and Total omega. I was reading about the outputs, but I find nothing about ...
10
votes
2answers
187 views

Identification of useless questions from a questionnaire

I'm developing a questionnaire. To improve its reliability and validity I want to use statistical methods. I want to eliminate questions whose answers are always the same. This means that nearly all ...
11
votes
3answers
3k views

Factor analysis of questionnaires composed of Likert items

I used to analyse items from a psychometric point of view. But now I am trying to analyse other types of questions on motivation and other topics. These questions are all on Likert scales. My initial ...
1
vote
1answer
5k views

How to calculate validity of questionnaire in SPSS?

I have 3 types of questionnaires, A 15 item scale with each item on a 7-point Likert type scale. Six subscales each containing different number of items 21 subscales each containing 4 items. ...
0
votes
2answers
50 views

Can I turn a latent variable be treated as an observed variable?

I am a doctoral candidate and for my dissertation research I am using two standardized scales as independent variables. Because these have not been used with my particular population, I needed to do ...
4
votes
3answers
216 views

Are discrimination parameters in two-parameter IRT models only specific to items?

Discrimination parameters in two-parameter model from IRT are usually considered item parameters. But I've come to doubt it. Think about psychophysics; for example, detecting luminance. I don't ...
14
votes
4answers
4k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
11
votes
3answers
3k views

Validating questionnaires

I am designing a questionnaire for my dissertation. I am in the process of validating the questionnaire I have applied a Cronbach's alpha test to the initial sample group. The responses to the ...
4
votes
4answers
148 views

If a measure is valid (but not necesarily reliable), can it be consistently replicated?

I had this question on an exam, and I was positive that the answer was A. The Beck Depression Inventory is a scale intended to measure depression levels, with higher scores indicative of higher ...
5
votes
1answer
610 views
4
votes
1answer
209 views

Good internal factor structure but poor cronbach alpha?

I am running a CFA and getting good fit indices (CFI = .99, RMSEA = .01) for a uni-dimensional scale. However, when I test for internal consistency, I get poor Cronbach alphas (alpha = .6). I have ...
0
votes
1answer
52 views

Several items for one construct

I am a dummy in statistics but need to accomplish some simple survey analyses. My question is: if I have several questions (items) for one construct (say, questions JS1-JS3 for job satisfaction) with ...
2
votes
2answers
2k views

Likert scale and yes/no calculation of one score

My survey is based on observations and interviews. The observation questions are yes/no type questions and interview questions are 4 points. $1=$ never, $2=$ sometimes, $3=$ usually, and $4=$ ...
0
votes
1answer
71 views

Calculating total score for a scale including polytomous and dichotomous items [duplicate]

I have a variable that is measured using a 13-item, 4-point Likert scale with the exception of one item (yes $=1$, no $=2)$. How can I calculate the total score for this variable?
1
vote
3answers
750 views

Is there an R package which implements weighted maximum likelihood for Rasch models?

Is there a package in R which implements the weighted maximum likelihood method (Warm, 1996) for estimating the person parameters in Rasch Models?
1
vote
0answers
33 views

How to calculate differential item functioning when factor structure differs between groups?

I want to test for differential item functioning on a self-report measure between two groups (i.e., one with a disease one without). Differential item functioning refers to a measurement bias wherein ...
1
vote
0answers
147 views

Intra- and inter- observer error, and the intra-class correlation coefficient (ICC)

I have 3 observers, who each perform multiple observations in which they count a phenomenon. I'm trying to quantify both inter- and intra- observer error for this study. I'm new to this and am finding ...
10
votes
5answers
1k views

Is it appropriate to treat n-point Likert scale data as n trials from a binomial process?

I've never liked how people typically analyze data from Likert scales as if error were continuous & Gaussian when there are reasonable expectations that these assumptions are violated at least at ...
1
vote
0answers
31 views

Map a normal distribution N(x,s) to an ordinal response variable

I am working on a simulation. I am going to extract a series of normally distributed values from a distribution N(x,s) whose mean x and variance s is known. I want to pretend that such distribution ...
3
votes
3answers
1k views

A multitrait-multimethod matrix and data set

I am working my way through Multitrait-Multimethod Matrix in a psychometrics class. We're only required to be able to analyze them but I'd really like to be able to construct them. I think I am able ...
4
votes
3answers
141 views

How to check psychometric type of scales?

It is well known, that there are diffrent types of scales (see Wikipedia Level of measurement). In psycholinguistics there is Semantic Differential technique. It uses a scale (usually 5 or 7 grades) ...
3
votes
1answer
189 views

What is the correct use of Cronbach's alpha?

I am trying to use Cronbach's alpha in order to see if some variables are measuring the "same", but I have some doubts about the correct use: Can I compute alpha for numerical, ordinal or ...
1
vote
1answer
241 views

Cronbach's alpha vs. KR20/KR21 in reliability studies

Should we calculate Cronbach's alpha instead of KR20/KR21 for measuring the reliability of an achievement test, or a test with dichotomous data?
3
votes
0answers
89 views

Should I run measurement invariance if confirmatory factor analysis is not affirmative?

I have a repeated measures design for a Likert scale with 10 items. This scale is assumed to be unidimensional and this is one of the research questions along with the scales measurement invariance ...
9
votes
2answers
774 views

Where do the descriptors for Cronbach's alpha values come from (e.g., poor, excellent)?

It seems fairly common to describe Cronbach's alpha values as follows: α ≥ 0.9 Excellent 0.7 ≤ α < 0.9 Good 0.6 ≤ α < 0.7 Acceptable 0.5 ≤ α < 0.6 Poor α < 0.5 Unacceptable Where ...
1
vote
0answers
89 views

Confirmatory factor analysis for a newly constructed questionnaire

I am about to develop a new questionnaire with 142 questions I have performed exploratory factor analysis and 30 latent factors were extracted. I was wondering if it is necessary to perform ...
8
votes
2answers
2k views

How to reduce number of items using factor analysis, internal consistency, and item response theory in conjunction?

I am in the process of empirically developing a questionnaire and I will be using arbitrary numbers in this example to illustrate. For context, I am developing a psychological questionnaire aimed at ...
1
vote
0answers
180 views

Doing CFA on a known theoretical model, but having problems with convergent and discriminant validity

I did a survey based on the UTAUT model / theory with standard questions used. The N was 150. I am trying to validate the model for my survey with structural equation modeling (via the AMOS software). ...
5
votes
2answers
114 views

Sampling considerations in psychometric applications of item response theory

Background I have developed a recent interest in Item Response Theory and its applications. I am studying clinical psychology and am most interested in polytomous models aimed at modelling ...
3
votes
1answer
197 views

How to apply published factor analysis loadings to sample data to compute factor scores?

I have data on 18 participants responding to a 30-item instrument. I also have a paper in which the researchers performed a principal axis factor analysis on the same survey where they report the item ...
3
votes
1answer
138 views

Item response theory vs. summative Likert scale

If the items to be "summed" or combined to create an overall index are collectively the underlying construct (i.e. I am trying to measure compliance to an intervention which has different ...
1
vote
1answer
102 views

Two treatments - How do I compute reliability estimates like Cronbach $\alpha$?

I randomized my survey, and now I have my data on SPSS (and all cleaned up) but when trying to run my Cronbach's alpha I have a problem. Because I have two treatments (sexualised and normal) all my ...
0
votes
2answers
93 views

Should I use raw scores or profile scores for variables in a regression model?

I'm trying to build a regression model to predict subjective memory reports based on the following predictors: objective memory test, illness perceptions, verbal fluency, and mood. I also want to ...
1
vote
1answer
134 views

Using existing questionnaire

I will be doing my colloquium and will be using existing questionnaires. I would like to know if there are authors/authorities who say that an existing questionnaire need not be validated. The ...
2
votes
1answer
316 views

Predicting dependent variables where independent variables are samples with differing sample size

Is there a standard method of dealing with independent variables, where they are samples from a known distribution, and the sample sizes differ from sample to sample? I'll give an example of what I ...
3
votes
1answer
109 views

Summing up ordered items to a sum-score: implications?

It is common practice in psychology to add together ordered items to sum-scores, and then use these sum-scores for subsequent statistics. This is usually done without explicitly mentioning measurement ...
2
votes
3answers
161 views

How can there be a standard deviation of true scores?

Dudek (1979) describes the following formula as "the standard deviation of true scores when the observed score is held constant" (p. 337): $SE_{est} = SD \sqrt{(reliability × (1 - reliability))}$ ...
3
votes
1answer
280 views

Cronbach's alpha negative result

What should I do if I am getting negative results for Cronbach's alpha? Is there anyway to adjust it?
3
votes
1answer
148 views

Why don't we square the reliability correlation when calculating reliable variance?

This question relates to situations where we apply some test to the same people at Time 1 and Time 2, and then calculate the correlation between scores at Time 1 and Time 2. I am reading the book ...
3
votes
0answers
332 views

Can I combine several ordinal questions to create an index and/or composite measure?

I am using a survey that contains several questions about various dimensions of performance for policy research institutes. Here, performance in the policy arena is unpacked into things like: ...
2
votes
1answer
223 views

IQ adaptive test items in 1pl, 2pl or 3pl IRT model?

Some adapative test systems (e.g. school assessment tools) use the 1pl IRT model, while others use the 2pl or the 3pl. When developing an adaptive IQ test, is there a rule of thumb about which model ...