Questions tagged [psychometrics]
Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.
309
questions
1
vote
0answers
26 views
How to assess agreement among measurement models (factor analysis) from multiple studies?
I've been collating studies that apply factor analysis on likert-like scale surveys for a specific psychometric construct. The original speculation from source material is that there should be 5 ...
2
votes
1answer
62 views
Inclusion of a NA category in inter-rater agreement calculation
I have a dataset in which the behaviour of $n = 6$ subjects have been rated by $k = 84$ raters on an ordinal scale with 5 ordinal categories. The distances between the ordinal categories are not ...
0
votes
0answers
15 views
Implicit for loops in PyMC3 using likelihood of matrices?
I'm confused about (for lack of a better word) "implicit for loops" in PyMC3. You'll note below that I define a 5x5 matrix, k. This matrix represents whether a specific child answers a ...
2
votes
1answer
42 views
How can I compare the variance or correlation between ranked *single answer* polls?
I am an undergrad psych major examining the differences in personal preferences between the extremist website 4chan and other websites, which will represent the pop norm. I would like to test whether ...
1
vote
1answer
58 views
Bayesian Multiple Item Response Models?
I am trying to extend the ideas of item response theory to multiple responses. Consider a marketing survey, which asks customers, "what's the deciding factor in whether or not you purchase ...
6
votes
1answer
47 views
What is the role of non-anchor items when equating scales across studies using IRT?
I'm trying to use IRT to equate scales across 5 datasets.
Three of them use the same scale (A); the other ones use different ones (B and C)
There are items that overlap between A and C, A and B, but ...
2
votes
0answers
37 views
Are there supposed to be bounds on parameters in 2PL Item Response Theory models?
Recently I've been studying Item Response Theory (IRT) and have come across some issues with the application side of it. I currently have a dataset of ~200 respondents x 7405 questions (quite ...
0
votes
0answers
13 views
Questions about post-hoc correlations and their legitimacy
I have a questions regarding correlations. Let me start with a quick background. I have conducted a Master research project. We recruited high trait anger men which we divided into two groups. One ...
0
votes
0answers
10 views
interpretation of item tree analysis plot
Two quick questions about Item-Tree analysis plot.
(1) For the red plot, I know items {1L, 3L, 5L, 6L} are equally informative and thus not plotted. Does this mean ...
4
votes
1answer
54 views
Do 1PL IRT models measure both ability and difficulty, or just difficulty?
I'm trying to better understand Item response Theory (IRT) from a Bayesian perspective. Hypothetically, suppose I want to use a 1PL model and my data is a binary matrix
...
0
votes
2answers
23 views
Is this an instance of convergent evidence?
In psychometrics, convergent validity is a source of proof for construct validity.
It is said that it is the degree to which test scores are correlated with test scores of relevant constructs or the ...
0
votes
0answers
9 views
calculating standard scores from skewed raw scores
I'm trying to create norms from raw scores of a psychological test. The raw scores are essentially number of items correct. They are skewed and their distribution is best described by a sinh-arcsinh ...
0
votes
0answers
13 views
0
votes
0answers
17 views
How much time between testing for comparison of survey-methods
Consider one came up with a new surveying method, for example an application that relies on some form of gamification of question-answering and wanted to compare how participants would use it to ...
0
votes
1answer
64 views
IRT: What is enough information? And can a test provide too much?
I was wondering what makes for a good test information value in IRT, and came across this:
Reliability: Item and test information functions graphically reflect
how reliably the individual items and ...
0
votes
0answers
13 views
A statistical test for a single factor model applied to factor loadings in mulitple factor analyses?
I have an number of factor analyses that all have been solved with the same number of factors. Is there a statistical test (in R?) that could determine whether or not the factor loadings are ...
1
vote
0answers
16 views
Non-independence of trial likelihoods in a staircase procedure?
In psychometrics,
we often want to know, for instance,
a given participants' perceptual threshold:
the intensity of a stimulus that they can detect 50% of the time.
It's common to use staircase ...
0
votes
1answer
40 views
Calculating Theta scores for identified subjects (using R package ltm) [closed]
I am using the R package ltm to calculate a 2PL IRT model.
goal:
creating a list of subjects IDs with the calculated Theta scores.
problem: ltm requires that the input includes only item scores, and ...
1
vote
1answer
76 views
What is the difference between Factor Analysis and Path Analysis in Structural Equation Modelling?
I am still a beginner in Structural Equation Modelling and I don't have a solid background in Statistics, so I apologize if my question is a bit naïve.
What I traditionally do when creating a ...
7
votes
1answer
284 views
Analyzing a partially crossed design
I have a data set from a test (see below). The scoring algorithm gives each item (item_id) a score (y) that is continuous from $...
3
votes
0answers
42 views
Does it make sense to use the Spearman-Brown correction (prediction / prophecy) formula with Kendall's Tau correlation coefficient?
The Spearman-Brown prophecy formula is used to estimate the increase in reliability that is obtained when more measurements are taken (and aggregated) from the same construct. Reliability is the ...
0
votes
0answers
15 views
Trying to compute factor scores from second-order EFA analysis - non-conformable arrays error
I'm stuck at the last point of my analysis where I am trying to compute factor scores from my factor analysis.
I get the following error, but I've checked that the number of rows/cols of the matrix ...
0
votes
0answers
12 views
Choosing a solution from Exploratory Factor Analysis
I have a survey with 337 responses to 31 questions (variables). All questions are on the 7 point Likert scale and I am trying to do a factor analysis.
First I tried doing Confirmatory Factor Analysis ...
1
vote
0answers
38 views
How to use Cronbach's Alpha?
Is it appropriate to use Cronbach's Alpha for nominal variables? I have 20 nominal variables (two dimensions with 10 questions each) that ask about the characteristics of work performance.
The ...
0
votes
0answers
17 views
Is it advisable to dichotomize an independent variable measured using a Likert scale? [duplicate]
I am working on a research project to identify the relationship between sexual esteem and sadistic behavior. My independent and dependent variable have both been measured using standard questionnaires ...
3
votes
0answers
21 views
How to deal with data from different centers?
I want to examine the psychometric properties (e.g. internal consistency, factorial validity) of a scale. The data (N=700) comes from 16 different locations. Central means and variances did not differ ...
0
votes
0answers
10 views
Pointers on how to proceed with my survey-method based research
I am doing a research on the effects of dominant personality and sexual self esteem on an individual's need for sexual dominance. I have decided to use the survey method for the project using two ...
1
vote
0answers
29 views
Performing a CMH test in R [closed]
I'm trying to perform a CMH test to examine differential item functioning by sex in R. I've tried cmh.test and mantelhaen.test ...
0
votes
1answer
26 views
Why can exploratory factor analysis (EFA) suggest a single factor, yet a 1-factor confirmatory factor analysis (CFA) model not fit?
I know I previously saw a thread on this topic, but I cannot locate it. So, please forgive my redundancy.
I have an 8-item questionnaire administered to a (small) sample of 59 patients. The data ...
0
votes
1answer
23 views
Can I have 2 Cronbach's alphas?
I've got an overall Cronbach's alpha of -.3. The questions are reverse scored. When I calculate 5 out of the 10 questions which have high to low scoring, alpha is .8, and when I find alpha for the ...
0
votes
0answers
22 views
Internal consistency: Alternative measures for unidimensional scales with a small number of items?
I use r and analyze a scale of 11 items, theoretically assuming one global factor and three group factors. Since a CFA revealed numerous correlated errors and due to the criticism of cronbachs alpha (...
0
votes
1answer
34 views
Inferential interpretation of Kappa reliability
I have obtained a Fliess' Kappa interrater reliability of $37.5\%$ from $90$ randomly selected raters rating $100$ randomly selected, binary items ($Yes/No$).
Question: Is the following ...
2
votes
1answer
42 views
How to intrepret one significant and one insignificant mediator and a significant direct effect in PROCESS?
I have conducted a mediation analysis with two mediators using PROCESS.
My findings are that in X -- a --> M1 -- b --> Y, a and b are significant.
However in X -- c --> M2 -- d --> Y, only d is ...
0
votes
0answers
6 views
How to dermine if a test is standardised?
What would you say, when can you assume that a test is standardised? With Standardised tests you can assume a continuous scale, which makes a lot more calculations possible. However, often times I am ...
1
vote
0answers
133 views
How to test for unidimensionality using the mirt r-package?
As it is my understanding that the ltm-package is obsolete, I'm looking for an equivalent to ltm::unidimTest() in the mirt-...
1
vote
0answers
30 views
Are there any simple pairwise comparison tests suitable for comparing emotions?
So, I'm pretty new to using statistics in a research context. I am currently writing my thesis in didactics of mathematics. In my study, a group of $10$ students are to solve two different ...
1
vote
0answers
76 views
Cronbach's Alpha with repeated missing values
I would like to use Cronbach's Alpha to check the fiability or internal consistency of a cognitive task. The task has a certain number of trials which were coded with zeros (incorrect trial) and ones (...
0
votes
1answer
47 views
Is signal detection theory a Bayesian approach?
Signal detection theory can be applied to modeling perception thresholds for defined stimuli, for example the detection of the presence of a specific signal. It is assumed both the stimulus intensity ...
0
votes
0answers
23 views
Relationship between people's insight and behavior
My main interest is to see how well people can identify the reasons behind their decisions. To measure this, I have collected 2 types of data: one set of variables that measures how important people ...
0
votes
0answers
24 views
Generalizability study of subscaled rating instrument
I have a school rating ($1$ thru $4$) instrument that consists of $9$ subscales (e.g., classroom instruction, school management etc.). Under each subscale, I have $6$ items on which rating occurs.
I ...
0
votes
1answer
105 views
lmer fixed effects coefficients: which is the term of comparison for levels? [closed]
I have a lmer model with two within factors
lmer (variable ~ time * condition + (1|id), data= df)
with time having 2 levels (<...
0
votes
1answer
36 views
questionnaire item exclusion criteria
I have compiled a questionnaire consisting of items which aim to measure different aspects of elevator and staircase use.
The main idea when composing the items was that it would be the most ...
2
votes
1answer
38 views
Item sampling techniques to reduce processing time for questionnaire
I have a well-crafted draft of a questionnaire, containing 240 items. The average processing time is 20 secs per item, so the entire questionnaire would take around 1 hour 20 min to complete. I want ...
4
votes
1answer
63 views
Difference between “distractors” and “foils”
What is the difference between "distractors" and "foils" in experimental design in psychology?
0
votes
1answer
66 views
Correlations - multiple observations & multiple observers
I haven't found quite this question on here.
I have a data set that typically requires generalizability theory to untangle however I now face a question I believe g theory can not answer.
17 ...
0
votes
1answer
980 views
item-total correlation vs. inter-item correlation
I am assesing the reliability and validity of an 16-item multuiple choice test supposed to test language skills. Cronbach's apha is "questionable" with .65. I am having troubles interpreting the ...
2
votes
1answer
78 views
3
votes
1answer
63 views
Combining multiple response models
Within a particular instrument, is it valid to combine multiple response models for subsets of items when estimating a latent trait?
As an example, if I had some items that were conceptually suited ...
0
votes
0answers
1k views
Proper way to plot confidence interval under normal distribution curve?
I write a small program which calculates a confidence interval for a given norm value. The output is then visualized by plotting the confidence interval and the corresponding hypothetical norm value ...
2
votes
2answers
277 views
How do I calculate d' from experimental data?
In Signal Detection Theory, d' is defined by the z-scores of Hits and False Alarms:
d' = z(Hits) - z(False Alarms).
Say the task is to detect if a certain object is present in a series of pictures, ...
