Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

2
votes
1answer
24 views

Cronbach's alpha vs. Average Item-Sum correlation?

I have heard that Cronbach's alpha is used to measure reliability of a test (to check if a set of items/question are measuring the same construct). How does the Cronbach's alpha compare with the ...
4
votes
1answer
299 views

Omega H asymptotic

In the omega function in the psych package for R, we have several outputs including Hierarchical omega, Asymptotic omega and Total omega. I was reading about the outputs, but I find nothing about ...
0
votes
0answers
17 views

Scaled vs. unscaled Chi-square statistic in CFA / GRM fit evaluation

I generally want to assess the fit of a CFA model with ordered categorical indicators (the graded response model, GRM). There are two versions of this model, the normal-ogive GRM (ordered CFA with ...
1
vote
1answer
12 views

Randomization scheme for rating images and assessing inter-rater reliability

Apologies for the basic question, but I'm new to the area and I'm working on a research project on rating images, and assessing the reliability of these ratings, with the following structure: There ...
0
votes
1answer
26 views

What is a design matrix? How do I build a design matrix for Rasch analysis?

I am trying to do an LRSM Rasch analysis of 72-item data. This data has 67 binary items and 5 ternary items. I need to create a "design matrix" in order to analyze this data. I do not really know what ...
4
votes
4answers
388 views

Are discrimination parameters in two-parameter IRT models only specific to items?

Discrimination parameters in two-parameter model from IRT are usually considered item parameters. But I've come to doubt it. Think about psychophysics; for example, detecting luminance. I don't ...
6
votes
1answer
405 views

How high is too high with Cronbach's alpha?

Someone in my organization just sent me a scale they developed (modified items from five separate scales used in published research) to measure an employee's readiness for change. They tested the ...
3
votes
1answer
243 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
1
vote
1answer
694 views

Doing CFA on a known theoretical model, but having problems with convergent and discriminant validity

I did a survey based on the UTAUT model / theory with standard questions used. The N was 150. I am trying to validate the model for my survey with structural equation modeling (via the AMOS software). ...
4
votes
1answer
703 views

What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
2
votes
2answers
471 views

How is Item Response Theory (IRT) called for continuous response?

I would like to model my problem using something similar to Item Response Theory - but my responses are not binary - they are continuous in [0;1]. How are these models/the research field called?
0
votes
0answers
30 views

convergence warning in glmer(),

Ok I have a dataset which is looking at choice behavior of rats in an olfactory decision-making task. In this task the rat has to sniff at a center odor port and depending on which odor stimulus it ...
2
votes
2answers
101 views

What is the two alternative forced choice paradigm

Is a two alternative forced choice paradigm (2AFC) an experimental design?
1
vote
1answer
64 views

What does 'downward pressure on sample size' mean?

In this article, there is this phrase 'downward pressure on sample size'. This is the paragraph: no matter the approach to collecting this new data, when attempting to ...
2
votes
1answer
17 views

Correspondence between two different depression scales?

I have become involved in a project where psychiatrists are studying the correspondence between two scales measuring the level of depression among a specific patient group. One of the scales is for ...
0
votes
0answers
24 views

Test-retest reliability

I have come across following table in a publication, where r is correlation coefficient. I could not find detailed procedures for test-retest reliability, is it any different from paired t-test ? ...
0
votes
0answers
35 views

Average inter-item correlations as internal consistency indicator

In literature, the average inter-item correlations in the range of 0.15 to 0.50 is considered as an indicator for an acceptable level of consistency. Why do the high average inter-item correlations ...
1
vote
0answers
34 views

How can measure be unreliable but valid?

Some previous research found evidence for the predictive validity of a measure. I've done some work and found the measure to be unreliable. Since reliability provides an upper limit for predictive ...
0
votes
0answers
8 views

How to form multi-item scales after doing a factor analysis

I am doing a content analysis research for bringing out news frames, and decided use Semetko and Valkenburg (2000) method, which consists in developing a set of questions that will help measure ...
0
votes
0answers
24 views

How to equate the cut scores

I have two different tests with two different cut scores. For example a test has a cut score of 50 and the other test has a cut score 55. I want to equate the cut scores for two test. I have ...
4
votes
2answers
92 views

The valid variance components to include in a generalizability theory model (anova) involving nesting? Are these correct interactions?

My question is about generalizability theory. A study by Bergeron, Floyd, McCormack, and Farmer (2008) is only adding to this confusion. Basically, in G-theory you want the first model you run to ...
0
votes
0answers
12 views

comparing SEM in Z and T scores

I have a function that gives me the SEM of my test for different thetas. This - I believe - is a SEM of Z scores. ...
1
vote
0answers
26 views

how to remove outliers prior to multiple imputation

A colleague came to me with the following problem. She has a complex, multivariate data set, in which respondents completed a number of measures with anywhere from 6 to 30 Likert type items for each ...
0
votes
0answers
36 views

How to calculate discrimination index and difficulty index of short answer test items?

How to calculate discrimination index and difficulty index of short answer test items? Should I use t test or the formula upper group score - lower group score divided by number of students in a ...
0
votes
1answer
77 views

Detecting Differential Item Functioning (DIF) using ordered logistic regression

I am in a situation in which I want to investigate whether test items of a questionnaire appear to have DIF. The items are polytomous and ranges from 1 to 5, with 5 being the most positive answer. ...
1
vote
0answers
20 views

Select and weigh questions on a reading assessment

I am a phd student in computer science and as such I am the goto guy for anything "mathy" in the cross discipline research group to which I belong. I have recently been given an assignment at work I'm ...
1
vote
0answers
7 views

Inter-raters reliability for sentiment analysis lexicon

I have a list of words (more than 2000) which were rated by 4 raters for sentiments like anger (scale from 1 to 5). I run an alpha test on SPSS which gives me a coefficient of >8 which shows a good ...
1
vote
1answer
43 views

raising the probability of correct answers of nextItem (catR)

Using the catR package I am trying to create a nextItem call that respondents would answer correctly about 65% of the time (even if that means the method is not as efficient as it could be). Children ...
0
votes
0answers
22 views

Calculate within and between group correlation

MY DATA: A surveys which contains 2 different questionnaires on the same subject which were taken by n=800 patients (i.e., timepoint) (#1 questionnaire: 5 questions which reach a total score of 0-42, ...
0
votes
0answers
24 views

Statistical Method for Normalization of Data

I collected data using a pre-test instrument to capture baseline emotions (two dimensions: valence and arousal) of participants. The two dimensions in the questionnaire range from calm to excited ...
3
votes
0answers
22 views

Comparing confirmatory factor analysis models with different number of observed variables

I have a 14 item scale that is hypothesized to measure self-esteem. The first model I specify uses all 14 items and predicts one latent factor (i.e., self-esteem). I would then like to specify a ...
4
votes
1answer
104 views

What is the difference between formative and reflective measurement models?

Can anyone provide examples as to why choose one over the other? Are they calculated diferently?
7
votes
4answers
16k views

What are the primary differences between z-scores and t-scores, and are they both considered standard scores?

We are currently converting student test scores in this manner : ( ScaledScore - ScaledScore Mean ) / StdDeviation ) * 15 + 100 I was referring to this ...
2
votes
3answers
237 views

“Two stage” factor Analysis: factoring saved factor scores

I have a model which consists of 19 questions, which are divided in three factors (factor 1 - nine questions, factor 2 - six questions, factor 3 - four questions). For this I did a factor analysis and ...
1
vote
0answers
16 views

CFA model with systematic temporal correlation between measures

I'm looking to fit CFA models to data in the case where measures have systematic temporal correlations between them. I'm familiar with autoregressive approaches, such as those described on Dave ...
3
votes
1answer
52 views

Non-Significant difference between individual items, but significant difference between overall scores

Say that we have a scale measuring Healthiness of Diet, and want to see if two populations have different levels of Healthiness of Diet. If we find that the difference between the two samples at the ...
3
votes
3answers
3k views

A multitrait-multimethod matrix and data set

I am working my way through Multitrait-Multimethod Matrix in a psychometrics class. We're only required to be able to analyze them but I'd really like to be able to construct them. I think I am able ...
7
votes
4answers
2k views

Validating an existing questionnaire into another language

I am doing research and trying to use a questionnaire that is available in English in another language. Is there a systematic approach for validating a questionnaire in another language? What ...
9
votes
3answers
233 views

Is it always better to extract more factors when they exist?

Unlike principal components analysis, the solutions to factor analysis models are not necessarily nested. That is, the loadings (for example) for the first factor won't necessarily be identical when ...
2
votes
1answer
74 views

Is empirical reliability in IRT the counterpart of internal consistency in CTT?

In IRT it is not meaningful to compute internal consistency coefficients such as Cronbachs alpha, because standard error of measurement (and thus measurement precision) is not assumed uniform across ...
2
votes
0answers
26 views

What measure of test reliability is appropriate if the final test output is not the sum of item scores?

I would like to compute a measure of reliability in the sense of Cronbach's alpha, McDonalds omega or glb. However it seems to me that all these measures assume that the final test score equates to ...
0
votes
0answers
54 views

Acceptable values for discrimination parameter - 2PL IRT Model

What is the acceptable range for discrimination parameters with the two-parameter IRT model? I have information from Baker (2001) but would love additional information. I have heard discrimination ...
1
vote
0answers
42 views

How can Bayesian inference improve upon logistic regression in incorporating psychometric data?

In order to estimate probability of default, banks and other financial institutions have often used logistic regression based on data involving credit history. Suppose an entrepreneur or borrower ...
0
votes
0answers
14 views

Combining different measures from different studies

I'm interested in combining outcomes from different psychiatric studies. However, the studies look at different types of diagnoses, that are measured sometimes with the same measurement, and sometimes ...
6
votes
2answers
2k views

Good internal factor structure but poor cronbach alpha?

I am running a CFA and getting good fit indices (CFI = .99, RMSEA = .01) for a uni-dimensional scale. However, when I test for internal consistency, I get poor Cronbach alphas (alpha = .6). I have ...
1
vote
1answer
433 views

Using Ltm package to calclate 3PL some lines of data seem to go missing

I am hoping someone can help with the following: I have been trying to use the ltm package in R to get person fit estimates using a 3PL model. The problem I have been having is that I start off ...
4
votes
1answer
112 views

What is Cronbach's Alpha intuitively?

I'm trying to understand Cronbach's Alpha intuitively. What is the general idea behind this construct? What properties were they trying to ensure that it had?
3
votes
0answers
31 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
1
vote
1answer
24 views

Can the test information function be used for a field test item bank?

I am field testing a few forms of items (with embedded anchor items) and will be analyzing the items using IRT, a 2PL model. These items will likely be used to create an item bank that can then create ...