Questions tagged [psychometrics]

Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

Filter by
Sorted by
Tagged with
2 votes
1 answer
26 views

Chi2 or Kruskal Wallis?

I have 3 different groups based on their level of resilience : low (n=60), moderate (n=35), high (n=10). After a qualitative thematic analysis of their responses, different subjects emerged, like &...
PsyCli's user avatar
  • 21
0 votes
0 answers
14 views

When using different groups of examinees to answer the same set of items, whether the item parameter should be invariant?

I am now doing a project to estimate the item parameters from the pharmacotherapy exams. Two groups of students from the same university answer the same set of items. From my understanding, the item ...
Cathy's user avatar
  • 1
0 votes
0 answers
12 views

Double bootstrap or 2 separated bootstraps to estimate the standard error?

I want to estimate the standard error of an estimate. It depends on the rows and columns of a dataframe. Two possibilities: Make a bootstrap on the rows to estimate $se_{row}$. Do the same with the ...
Julien's user avatar
  • 121
0 votes
0 answers
3 views

Aligning scores between the Child Behaviour Checklist (CBCL) and The Brief Infant-Toddler Social and Emotional Assessment (BITSEA)

I am currently conducting a longitudinal secondary data analysis. Due to the nature of the data that has previously been collected and my proposed analysis (Growth Mixture Modelling), I am needing to ...
AFD's user avatar
  • 1
0 votes
0 answers
11 views

How to deal with missing data in DIF analysis?

I would like to do a DIF analysis on a scale The problem is that I have missing data for 10 items for the focal group The questionnaire in question exists in two forms: a 15 item version and a 25 item ...
Ruben's user avatar
  • 1
0 votes
0 answers
31 views

Closed form relationship between phi coefficient and tetrachoric correlation?

I am (1) generating bivariate normal deviates $(Y_1, Y_2)$ (2) dichotomizing one of the normal deviates according to quantiles of N(0,1) to yield the normal, binary pair $(Y_1, Y_2^*)$. I want to ...
Emma Jean's user avatar
  • 605
0 votes
0 answers
13 views

Is confirmatory factor analysis a way to validate a scale?

I am doing a translation and validation of a scale. I do not have any other measures, so I cannot do convergent/discriminant validity assessments. Is CFA enough to validate a scale, and if not what ...
Shabby Chic's user avatar
0 votes
0 answers
30 views

False positive correction categorical data

I am trying to understand how to correct for chance categorisation as either impaired/not impaired across a range of measures. Briefly, I am looking to summarise prevalence statistics across ~40 ...
acerhelia's user avatar
0 votes
0 answers
18 views

Obtaining comparable scores across factors from factor analysis in R

I am looking for some help with factor analysis in R. I ran a factor analysis using psych package and ended up with a four-factor model, which is fine. BUT... I want to use the results in further ...
ievasieva's user avatar
0 votes
0 answers
71 views

Graded response model and generalized partial credit model show very different discrimination values. Which one to trust?

I have data from N = 265 people for some 7-point Likert-items that I would like to investigate. I am interested in selecting the best items for a scale. The items all load onto the same factor. I have ...
mkks's user avatar
  • 63
0 votes
1 answer
36 views

Sanity checking survival model

I have some basic statistics foundations (Lean Six Sigma, Industrial Engineering in College), but I'm completely new to survival analysis, and relatively new to Data Science. So I'm looking to sanity-...
Leonardo Segura's user avatar
1 vote
0 answers
53 views

Confidence Interval for factor loadings of 2PL model

I estimated a 2PL model obtained for some dichotomous data presenting answers of 12 "yes/no" questions from about 200 individuals. Now I want to calculate the confidence intervals for the ...
mugdi's user avatar
  • 240
0 votes
0 answers
8 views

how can I assess internal consistency for an algorithmic classification?

Is there a way to assess internal consistency for an algorithmic classification based on a rating scale? Or other appropriate reliability metrics? For instance, let's say that based on a 6-item, ...
zephryl's user avatar
  • 200
3 votes
0 answers
139 views

How to perform exploratory factor analysis with complex survey data for ordinal and dichotomous variables?

I would like to perform an exploratory factor analysis in R with complex sample survey data involving ordinal and dichotomous variables. The svyfactanal() function ...
user156625's user avatar
1 vote
0 answers
25 views

Confirmatory factor analysis parameterization of the 2-PL item response model

For binary item response $i$ by person $j$, the 2-PL item response model is of the form: $$Response_{ij} \sim Bernoulli(p_{ij})$$ $$logit(p_{ij}) \sim \alpha_i(\theta_j - \eta_i)$$ Usually, $\alpha_i$ ...
socialscientist's user avatar
1 vote
1 answer
283 views

Is it acceptable/common practice to report IRT theta scores as a percentile?

I have been teaching myself IRT. I think I get it, however, I've not yet reached a point where I would say that I'm totally confident. In assuming that the latent trait is normally distributed with ...
faustus's user avatar
  • 235
0 votes
0 answers
25 views

Can I summarize different forms of intraclass correlation coefficients (ICCs) descriptively?

There are various forms of intraclass correlation coefficients (ICCs) that differ in terms of their "model," "type," and "definition" (Koo & Li, 2016). So my question ...
Matthias's user avatar
3 votes
1 answer
41 views

Classification Method with Small Data

I have the following problem. I have individual observations $y_i \in [0,1]$ on the response to an increasing stimulus called $x$. The stimulus is discretized as $x^1, x^2, \ldots , x^n$. Let's use ...
deps_stats's user avatar
  • 1,687
0 votes
0 answers
353 views

Is it correct to calculate Average Variance Extracted (AVE) and Composite Reliability (CR) using loadings from exploratory factor analysis?

In order to obtain evidence of convergence and discriminant validity of a scale, I used SPSS to run a Exploratory Factor Analysis; and used its loadings to calculate the Average Variance Extracted (...
charliept99's user avatar
1 vote
2 answers
147 views

Factor analysis with multiple responses to a question

I have a data relating to a questionnaire that takes form of a knowledge test, in which each question has multiple correct responses and the respondents can check all responses they consider to be ...
Kate's user avatar
  • 11
1 vote
0 answers
70 views

Which intraclass correlation coefficient (ICC) to use to verify the test-retest reliability of a questionnaire?

I want to check the test-retest reliability of a questionnaire. -A total of 300 adolescents from the same school (10 classes) participated in the study. -The instrument was applied in two moments with ...
dr_stats's user avatar
1 vote
1 answer
138 views

Analysing multiple-choice question with multiple answers

I recently ran an experiment with 4 conditions where participants got a multiple choice task to select words best describing a subjective sensation. Now I'm struggling to figure out which analytical ...
mothos's user avatar
  • 11
0 votes
0 answers
291 views

Is this documentation for the R 'psych' package ICC() function incorrect?

I am not a statistician, but from my reading about various types of intraclass correlation measures, ICC(2,1) (Shrout and Fleiss nomenclature) is a measure of absolute agreement sensitive to additive ...
Evan's user avatar
  • 145
2 votes
1 answer
156 views

What are typical approaches to express the reliability of a categorical scale?

I have categorical scales consisting of items with each three answer options: 1) very much; 2) somewhat; 3) never. What would be a suitable approach to express reliabilities for scales like this? I am ...
DomB's user avatar
  • 521
1 vote
1 answer
35 views

Psychometric test validation question

We are trying to validate a test for young children using IRT analyses (rasch/2pl). Models do not converge due to a floor effect. Teachers also took this assessment. Would it skew results if i ...
em123's user avatar
  • 11
0 votes
1 answer
434 views

Multiple Regression: Why is there range restriction in predicted values?

I have a built linear regression model whereby I have used scores on a personality survey to predict manager ratings of job performance. Both my predictor and outcome variables have a min and max ...
RMAkh's user avatar
  • 3
0 votes
0 answers
56 views

When to use "fixed number of components" in a PCA

The question is relatively simple (and stupid maybe). When do I use a fixed number of components (especially in analyzing a questionnaire). Is it useful when I have literature to a validated scale/...
Peter Silie's user avatar
1 vote
0 answers
249 views

Using Pearson Moment Correlation For Establishing Validity Between Two Questionnaires

I'm moderately new to psychometrics and test design, and I'm currently in the process of attempting to develop a questionnaire that measures soft skills (communication, collaboration, work ethic, ...
Jace's user avatar
  • 11
1 vote
2 answers
58 views

I want to test for a significant difference in the result (t-value or effect size) of two paired t-tests

I am testing the quality of a measurement construct, where I propose that Measure A is better (i.e. more responsive to situational changes) than Measure B is. I conduct a study in which I measure the ...
Niels's user avatar
  • 11
0 votes
0 answers
23 views

IRT models for repetitive items

I'm looking for literature about IRT models that fit for tests in which items are presented for each examinee more than once and/or different questions are asked for the same item. For example, in the ...
S H's user avatar
  • 1
0 votes
0 answers
204 views

Valid 2nd order SEM with multiple dependent variables

I'm working on an SEM problem, and I could use some second opinions! The diagram below shows the proposed SEM model, where I have a 4 latent constructs which are estimated themselves from 2 second-...
logisticregress's user avatar
0 votes
1 answer
62 views

Showing that estimating $Y$ from $X$ via linear regression and estimating $X$ from $Y$ via linear regression are not invertible

This is a follow up to the question Regression on population values?. In the theory of test equating (see, e.g., Test Equating, Scaling, and Linking by Kolen and Brennan (2014)), suppose we have a ...
Clarinetist's user avatar
  • 4,203
4 votes
2 answers
125 views

Regression on population values?

In the theory of test equating (see, e.g., Test Equating, Scaling, and Linking by Kolen and Brennan (2014)), suppose we have a group of examinees whom we randomly assign one of two exam forms, say ...
Clarinetist's user avatar
  • 4,203
2 votes
1 answer
332 views

What is the minimum acceptable correlation for creating a composite variable?

I have created a composite measure comprising two items (both 7-point Likert scale items), using the summed z score method. The correlation coefficient for these two items is 0.59. This seemed ...
user145791's user avatar
1 vote
1 answer
136 views

Assess overall question difficulty in Graded Response Model

I used graded response modelling (GRM) to assess the difficulty of items from a survey using a Likert scale from 1 to 5. The results show the difficulty of each item (i.e., 1-5), for each question. I ...
Alex S's user avatar
  • 11
1 vote
0 answers
29 views

Denotion of Cronbach's Alpha based on Polychorric Correlations

we assessed several 5 point Likert-Scale items in a questionnaire and are interested in Cronbach's $\alpha$ for the internal consistency. I recently read (here) that it can be more precise to used ...
SysRIP's user avatar
  • 73
4 votes
2 answers
118 views

When and why should psychologists use ordered probit models rather than the general linear model?

Psychologists often use the general linear model with ordinal independent/dependent variables (i.e. Likert scales to measure 'levels' of a psychological trait. For example, assigning numbers to the ...
Henry Ward's user avatar
1 vote
0 answers
91 views

How to assess agreement among measurement models (factor analysis) from multiple studies?

For a meta-analysis, I've been collating studies that apply factor analysis on likert-like scale surveys for a specific psychometric construct. The original speculation from source material is that ...
Galen's user avatar
  • 6,217
2 votes
1 answer
414 views

Inclusion of a NA category in inter-rater agreement calculation

I have a dataset in which the behaviour of $n = 6$ subjects have been rated by $k = 84$ raters on an ordinal scale with 5 ordinal categories. The distances between the ordinal categories are not ...
Dom42's user avatar
  • 85
3 votes
1 answer
56 views

How can I compare the variance or correlation between ranked *single answer* polls?

I am an undergrad psych major examining the differences in personal preferences between the extremist website 4chan and other websites, which will represent the pop norm. I would like to test whether ...
user avatar
2 votes
1 answer
83 views

Bayesian Multiple Item Response Models?

I am trying to extend the ideas of item response theory to multiple responses. Consider a marketing survey, which asks customers, "what's the deciding factor in whether or not you purchase ...
jbuddy_13's user avatar
  • 2,183
6 votes
1 answer
110 views

What is the role of non-anchor items when equating scales across studies using IRT?

I'm trying to use IRT to equate scales across 5 datasets. Three of them use the same scale (A); the other ones use different ones (B and C) There are items that overlap between A and C, A and B, but ...
llbia's user avatar
  • 331
2 votes
0 answers
138 views

Are there supposed to be bounds on parameters in 2PL Item Response Theory models?

Recently I've been studying Item Response Theory (IRT) and have come across some issues with the application side of it. I currently have a dataset of ~200 respondents x 7405 questions (quite ...
underscoresourcecode's user avatar
4 votes
1 answer
288 views

Do 1PL IRT models measure both ability and difficulty, or just difficulty?

I'm trying to better understand Item response Theory (IRT) from a Bayesian perspective. Hypothetically, suppose I want to use a 1PL model and my data is a binary matrix ...
jbuddy_13's user avatar
  • 2,183
0 votes
2 answers
115 views

Is this an instance of convergent evidence?

In psychometrics, convergent validity is a source of proof for construct validity. It is said that it is the degree to which test scores are correlated with test scores of relevant constructs or the ...
Joe's user avatar
  • 1
2 votes
1 answer
315 views

IRT: What is enough information? And can a test provide too much?

I was wondering what makes for a good test information value in IRT, and came across this: Reliability: Item and test information functions graphically reflect how reliably the individual items and ...
kh_one's user avatar
  • 357
1 vote
0 answers
26 views

Non-independence of trial likelihoods in a staircase procedure?

In psychometrics, we often want to know, for instance, a given participants' perceptual threshold: the intensity of a stimulus that they can detect 50% of the time. It's common to use staircase ...
Eoin's user avatar
  • 7,587
0 votes
1 answer
445 views

Calculating Theta scores for identified subjects (using R package ltm) [closed]

I am using the R package ltm to calculate a 2PL IRT model. goal: creating a list of subjects IDs with the calculated Theta scores. problem: ltm requires that the input includes only item scores, and ...
BoTz's user avatar
  • 113
2 votes
1 answer
1k views

What is the difference between Factor Analysis and Path Analysis in Structural Equation Modelling?

I am still a beginner in Structural Equation Modelling and I don't have a solid background in Statistics, so I apologize if my question is a bit naïve. What I traditionally do when creating a ...
cookiemonster1697's user avatar
7 votes
1 answer
823 views

Analyzing a partially crossed design

I have a data set from a test (see below). The scoring algorithm gives each item (item_id) a score (y) that is continuous from $...
Simon Harmel's user avatar

1
2 3 4 5
7