Skip to main content

Questions tagged [psychometrics]

Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

Filter by
Sorted by
Tagged with
0 votes
1 answer
29 views

Determining chance level for forced-choice performance validity tests

Psychologists, especially neuropsychologists, use performance validity tests (PVTs) to detect individuals who might intentionally underperform on ability tests, e.g., memory tests, because diagnosed ...
Mark D Worthen PsyD's user avatar
2 votes
1 answer
31 views

Why is reliability a ratio of true score variance to total score variance, instead of the ratio of true score to total score?

Reliability is given by: $r_{xx} = \frac{Var(T)}{Var(X)}$ where $T$ is the true score and $X$ is the total score. Why isn't reliability simply: $r_{xx} = \frac{T}{X}$ where $T$ would be a sum of all ...
J. Doe's user avatar
  • 357
0 votes
0 answers
32 views

How to measure correlation between anxiety level (score 20-80) and parenting style (authoritarian, authoritative, permissive)?

As the title says, I am wondering how to measure the correlation between anxiety level and parenting style, as one is ordinal and the other is categorical. Is Spearman's correlation the right method? ...
fruib's user avatar
  • 1
0 votes
0 answers
35 views

Factor Scores explain less Variance than Item means in a multilevel cfa

I conducted a multilevel confirmatory factor analysis (CFA) in lavaan on repeated measures questionnaire data (satisfaction with travel scale). 42 participants had ...
StatOru's user avatar
0 votes
0 answers
15 views

How to calculate reliability of difference scores?

I am trying to calculate the reliability of a difference score. Specifically, the data have, for each participant, scores for 10 items in Condition X (1s and 0s), as well as 10 different items in ...
Altair555's user avatar
2 votes
2 answers
76 views

Weighing Questions within a Survey

I work for a public child welfare agency. We have something called a level of care tool that categorizes children based on the amount of care they need. This helps us identify appropriate care options ...
ra_learns's user avatar
0 votes
0 answers
18 views

Score buckets from SED confidence intervals

I built an app to illustrate the reliability of standardized tests for a (somewhat) non-technical audience. I'm not a psychometrician, so I want some extra eyes on this to make sure it makes sense. ...
Inna-gadda-daavida's user avatar
0 votes
0 answers
29 views

What do bad infit and outfit in dichotomous Rasch model signify?

I'm really a beginner with Psychometry. We've just started psychometry in our college and I've used software Jamovi for that purpose. Working with Dichotomous Rasch models to asses a Multiple choice ...
Zuhair Al-Johar's user avatar
6 votes
2 answers
592 views

Why random noise is eliminable by infinite repetition of testing?

In the field of psychometry, Classical Test Theory "CTT" presumes that in a test the observed test score is the result of the true score + error score. The last is thought to raise due to ...
Zuhair Al-Johar's user avatar
0 votes
0 answers
58 views

Should I go with an unrotated factor analysis model?

I'm running a project on survey data where I have a bunch of very similar operationalizations of my DV (four different indices of my DV). Let's call it support for X behavior. All of them are ...
tank12758's user avatar
1 vote
1 answer
35 views

Factor Analysis: how to distinguish cross-loadings from correlated latent factors

Factor analysis can allow the factors to be correlated in 'oblique' rotations. But this seems underdetermined. How would (explanatory) factor analysis distinguish the two cases below (using the data ...
daaronr's user avatar
  • 131
1 vote
0 answers
30 views

CFA - How do within-factor error covariances affect how a psychometric scale is used/scored?

I'm working on a confirmatory factor analysis for a measure with one factor, 8 items, each is a 7-point Likert scale. Two items within the factor are worded very similarly and their errors are likely ...
JH22's user avatar
  • 11
1 vote
2 answers
96 views

Psychometrics - transferability of CFA and factor loadings to observed scores/sum scores in scale development

I have come across a situation in a dataset using a not very used scale, evaluated partly using confirmatory factor analysis (CFA). The CFA has provided support for a number of subscales. In some of ...
anna mariasson's user avatar
0 votes
0 answers
13 views

Equating between short form and long form of the same instrument

I have encountered several instruments in psychology and related fields where we have multiple versions of the same instrument, specifically a short form and a long form. For instance, the patient ...
Eric Boorman's user avatar
0 votes
0 answers
39 views

What is the test performed in this table?

Yesterday I was helping a friend in reviewing a clinical psychometric paper for his Bachelor's thesis. There, there is this table presented: Now, the paper states that (the emphasis is mine): "...
Alessandro Bertulli's user avatar
0 votes
1 answer
132 views

How to convert six point likert scale to four point likert scale

I would like to convert the mean score of a sample that was administered a questionnaire using a six point likert scale into a mean that is comparable to the same questionnaire using a four point ...
Aksel's user avatar
  • 1
0 votes
0 answers
35 views

Questions related to running an exploratory factor analysis on skewed data

I want to test overall skewness / normality in a large data set of ordinal data from survey questions. (I couldn’t use Shapiro wilk as I received an error saying the dataset is too large as it has ...
izzi3880's user avatar
0 votes
1 answer
53 views

Is ceiling effect mitigated by standardisation, renorming or combining with other indicators?

I have a psychological test that consist of a set of binary items. It has a noticeable ceiling effect in the sum scores (essentially, a majority of participants get most items correct). I also see ...
FredrikH-R's user avatar
3 votes
0 answers
44 views

Difference between kappa statistics

I am currently developing an instrument and have some questions on the use of kappa statistics for interrater agreement used for content validity. I have gone through previous literature and I still ...
Yip's user avatar
  • 31
3 votes
1 answer
35 views

Modeling within-person design (items-within-person), need to account for item?

Apologies if this is an elementary question - within-person designs are not my forte, and I am more used to designs in the quasiexperimental econometric tradition. I have two designs that are the same ...
RickyB's user avatar
  • 1,145
2 votes
1 answer
69 views

Chi2 or Kruskal Wallis?

I have 3 different groups based on their level of resilience : low (n=60), moderate (n=35), high (n=10). After a qualitative thematic analysis of their responses, different subjects emerged, like &...
PsyCli's user avatar
  • 21
0 votes
0 answers
15 views

How to deal with missing data in DIF analysis?

I would like to do a DIF analysis on a scale The problem is that I have missing data for 10 items for the focal group The questionnaire in question exists in two forms: a 15 item version and a 25 item ...
Ruben's user avatar
  • 1
0 votes
0 answers
89 views

Obtaining comparable scores across factors from factor analysis in R

I am looking for some help with factor analysis in R. I ran a factor analysis using psych package and ended up with a four-factor model, which is fine. BUT... I want to use the results in further ...
ievasieva's user avatar
0 votes
1 answer
93 views

Sanity checking survival model

I have some basic statistics foundations (Lean Six Sigma, Industrial Engineering in College), but I'm completely new to survival analysis, and relatively new to Data Science. So I'm looking to sanity-...
Leonardo Segura's user avatar
1 vote
0 answers
78 views

Confidence Interval for factor loadings of 2PL model

I estimated a 2PL model obtained for some dichotomous data presenting answers of 12 "yes/no" questions from about 200 individuals. Now I want to calculate the confidence intervals for the ...
mugdi's user avatar
  • 90
3 votes
0 answers
254 views

How to perform exploratory factor analysis with complex survey data for ordinal and dichotomous variables?

I would like to perform an exploratory factor analysis in R with complex sample survey data involving ordinal and dichotomous variables. The svyfactanal() function ...
user156625's user avatar
1 vote
0 answers
35 views

Confirmatory factor analysis parameterization of the 2-PL item response model

For binary item response $i$ by person $j$, the 2-PL item response model is of the form: $$Response_{ij} \sim Bernoulli(p_{ij})$$ $$logit(p_{ij}) \sim \alpha_i(\theta_j - \eta_i)$$ Usually, $\alpha_i$ ...
socialscientist's user avatar
3 votes
1 answer
698 views

Is it acceptable/common practice to report IRT theta scores as a percentile?

I have been teaching myself IRT. I think I get it, however, I've not yet reached a point where I would say that I'm totally confident. In assuming that the latent trait is normally distributed with ...
faustus's user avatar
  • 265
3 votes
1 answer
47 views

Classification Method with Small Data

I have the following problem. I have individual observations $y_i \in [0,1]$ on the response to an increasing stimulus called $x$. The stimulus is discretized as $x^1, x^2, \ldots , x^n$. Let's use ...
deps_stats's user avatar
  • 1,685
1 vote
2 answers
375 views

Factor analysis with multiple responses to a question

I have a data relating to a questionnaire that takes form of a knowledge test, in which each question has multiple correct responses and the respondents can check all responses they consider to be ...
Kate's user avatar
  • 11
1 vote
0 answers
92 views

Which intraclass correlation coefficient (ICC) to use to verify the test-retest reliability of a questionnaire?

I want to check the test-retest reliability of a questionnaire. -A total of 300 adolescents from the same school (10 classes) participated in the study. -The instrument was applied in two moments with ...
dr_stats's user avatar
2 votes
1 answer
232 views

Analysing multiple-choice question with multiple answers

I recently ran an experiment with 4 conditions where participants got a multiple choice task to select words best describing a subjective sensation. Now I'm struggling to figure out which analytical ...
mothos's user avatar
  • 21
0 votes
0 answers
444 views

Is this documentation for the R 'psych' package ICC() function incorrect?

I am not a statistician, but from my reading about various types of intraclass correlation measures, ICC(2,1) (Shrout and Fleiss nomenclature) is a measure of absolute agreement sensitive to additive ...
Evan's user avatar
  • 225
2 votes
1 answer
437 views

What are typical approaches to express the reliability of a categorical scale?

I have categorical scales consisting of items with each three answer options: 1) very much; 2) somewhat; 3) never. What would be a suitable approach to express reliabilities for scales like this? I am ...
DomB's user avatar
  • 531
1 vote
1 answer
38 views

Psychometric test validation question

We are trying to validate a test for young children using IRT analyses (rasch/2pl). Models do not converge due to a floor effect. Teachers also took this assessment. Would it skew results if i ...
em123's user avatar
  • 11
0 votes
1 answer
671 views

Multiple Regression: Why is there range restriction in predicted values?

I have a built linear regression model whereby I have used scores on a personality survey to predict manager ratings of job performance. Both my predictor and outcome variables have a min and max ...
RMAkh's user avatar
  • 3
0 votes
0 answers
65 views

When to use "fixed number of components" in a PCA

The question is relatively simple (and stupid maybe). When do I use a fixed number of components (especially in analyzing a questionnaire). Is it useful when I have literature to a validated scale/...
Peter Silie's user avatar
1 vote
0 answers
308 views

Using Pearson Moment Correlation For Establishing Validity Between Two Questionnaires

I'm moderately new to psychometrics and test design, and I'm currently in the process of attempting to develop a questionnaire that measures soft skills (communication, collaboration, work ethic, ...
Jace's user avatar
  • 11
1 vote
2 answers
69 views

I want to test for a significant difference in the result (t-value or effect size) of two paired t-tests

I am testing the quality of a measurement construct, where I propose that Measure A is better (i.e. more responsive to situational changes) than Measure B is. I conduct a study in which I measure the ...
Niels's user avatar
  • 11
0 votes
1 answer
53 views

IRT models for repetitive items

I'm looking for literature about IRT models that fit for tests in which items are presented for each examinee more than once and/or different questions are asked for the same item. For example, in the ...
S H's user avatar
  • 1
0 votes
0 answers
391 views

Valid 2nd order SEM with multiple dependent variables

I'm working on an SEM problem, and I could use some second opinions! The diagram below shows the proposed SEM model, where I have a 4 latent constructs which are estimated themselves from 2 second-...
logisticregress's user avatar
1 vote
1 answer
68 views

Showing that estimating $Y$ from $X$ via linear regression and estimating $X$ from $Y$ via linear regression are not invertible

This is a follow up to the question Regression on population values?. In the theory of test equating (see, e.g., Test Equating, Scaling, and Linking by Kolen and Brennan (2014)), suppose we have a ...
Clarinetist's user avatar
  • 5,027
5 votes
2 answers
172 views

Regression on population values?

In the theory of test equating (see, e.g., Test Equating, Scaling, and Linking by Kolen and Brennan (2014)), suppose we have a group of examinees whom we randomly assign one of two exam forms, say ...
Clarinetist's user avatar
  • 5,027
2 votes
1 answer
558 views

What is the minimum acceptable correlation for creating a composite variable?

I have created a composite measure comprising two items (both 7-point Likert scale items), using the summed z score method. The correlation coefficient for these two items is 0.59. This seemed ...
user145791's user avatar
1 vote
1 answer
298 views

Assess overall question difficulty in Graded Response Model

I used graded response modelling (GRM) to assess the difficulty of items from a survey using a Likert scale from 1 to 5. The results show the difficulty of each item (i.e., 1-5), for each question. I ...
Alex S's user avatar
  • 11
1 vote
0 answers
40 views

Denotion of Cronbach's Alpha based on Polychorric Correlations

we assessed several 5 point Likert-Scale items in a questionnaire and are interested in Cronbach's $\alpha$ for the internal consistency. I recently read (here) that it can be more precise to used ...
SysRIP's user avatar
  • 73
4 votes
2 answers
209 views

When and why should psychologists use ordered probit models rather than the general linear model?

Psychologists often use the general linear model with ordinal independent/dependent variables (i.e. Likert scales to measure 'levels' of a psychological trait. For example, assigning numbers to the ...
Henry Ward's user avatar
1 vote
0 answers
94 views

How to assess agreement among measurement models (factor analysis) from multiple studies?

For a meta-analysis, I've been collating studies that apply factor analysis on likert-like scale surveys for a specific psychometric construct. The original speculation from source material is that ...
Galen's user avatar
  • 9,084
3 votes
1 answer
684 views

Inclusion of a NA category in inter-rater agreement calculation

I have a dataset in which the behaviour of $n = 6$ subjects have been rated by $k = 84$ raters on an ordinal scale with 5 ordinal categories. The distances between the ordinal categories are not ...
Dom42's user avatar
  • 115
3 votes
1 answer
61 views

How can I compare the variance or correlation between ranked *single answer* polls?

I am an undergrad psych major examining the differences in personal preferences between the extremist website 4chan and other websites, which will represent the pop norm. I would like to test whether ...
user avatar

1
2 3 4 5
7