Questions tagged [psychometrics]
Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.
1
vote
1answer
13 views
Difference between “distractors” and “foils”
What is the difference between "distractors" and "foils" in experimental design in psychology?
0
votes
0answers
4 views
Convergent & Discriminant Validity -Same construct (measured by different meaures) in two different samples
I don't know how to go about looking at the Convergent & Discriminant Validity of the same construct measured with two different scales in two different contexts.
This is a cross-cultural ...
0
votes
0answers
16 views
Building a psychometric function for tasks using adaptivity paradigm [closed]
I'm trying to build a psychometric function via the Bayesian Estimate approach developed by Treutwein and Strasburger in Fitting the Psychometric Function: https://link.springer.com/article/10.3758/...
0
votes
0answers
6 views
Can I treat the Generalized Partial Credit Model as a scale in the Survey of Adult Skills (PIAAC)?
The Survey of Adult Skills (PIAAC) contains 250,000 rows and many columns. 6 of the columns contain the answer of questions in Problem-solving skills. Their possible values are:
The technical ...
1
vote
1answer
30 views
Correlations - multiple observations & multiple observers
I haven't found quite this question on here.
I have a data set that typically requires generalizability theory to untangle however I now face a question I believe g theory can not answer.
17 ...
0
votes
1answer
28 views
item-total correlation vs. inter-item correlation
I am assesing the reliability and validity of an 16-item multuiple choice test supposed to test language skills. Cronbach's apha is "questionable" with .65. I am having troubles interpreting the ...
2
votes
1answer
67 views
0
votes
0answers
13 views
How to test the validity of Fitt's law in R
I would like to use the Fitt's Law to test some hypothesis. Anyway, it's the first time I use it, and I have some doubts.
Then aim of my experiment is to test if in a certain situation the Fitt's law ...
0
votes
0answers
17 views
Comparing two IRT test information curves
I am fitting an IRT graded response model in R. I want to compare two measures in terms of at what levels each measure is most informative (e.g., is measure 1 informative at more extreme levels of the ...
0
votes
0answers
5 views
Comparison of discrimination tests using, Thurstonian model and its d -values
The task is to measure two discrimination test methods "triangle test and tetrad". In both methods the possibility to guess correct is (1/3).
In my study I used Chi square test to analyse data.
...
3
votes
1answer
43 views
Combining multiple response models
Within a particular instrument, is it valid to combine multiple response models for subsets of items when estimating a latent trait?
As an example, if I had some items that were conceptually suited ...
0
votes
0answers
142 views
Proper way to plot confidence interval under normal distribution curve?
I write a small program which calculates a confidence interval for a given norm value. The output is then visualized by plotting the confidence interval and the corresponding hypothetical norm value ...
0
votes
0answers
31 views
Coefficient omega: appropriate extraction of factors
I use omega_hiearchical to determine the internal consistency of a scale. If I understood it correctly, it tries to extract three lower level factors by default. But you can specify that you want to ...
2
votes
2answers
115 views
How do I calculate d' from experimental data?
In Signal Detection Theory, d' is defined by the z-scores of Hits and False Alarms:
d' = z(Hits) - z(False Alarms).
Say the task is to detect if a certain object is present in a series of pictures, ...
0
votes
0answers
15 views
Identifying problems in psychometric data set
I am analyzing the data from two questionnaires designed to measure the same trait which are highly correlated with one another and have very similar questions. My data is from one group which ...
3
votes
0answers
45 views
What is the intuition behind taking the sum of square roots, squared
In a recent publication, the authors report the following transformation when aggregating across three different scales:
Cognitive style level was used as a control variable and captured as
...
1
vote
0answers
12 views
Is it correct to a total Cronbach´s alpha analysis to a tridimensional scale?
There´s a statement that says that Cronbach´s alpha assumes unidimensionality. In that sense, is it correct to calculate a global alpha? I mean, by including all 3 dimensions, and not by separating ...
0
votes
0answers
19 views
How to interpret low fit of bifactor SEM model?
I am trying to evaluate the psychometric properties of a questionnaire instrument using bifactor SEM. A bifactor model with five group factors has the best fit, and this model has “good” RMSEA and ...
0
votes
1answer
25 views
Ceiling effect of data-IRT Rasch EFA
Can you do IRT, CFA and/or Rasch if your data has items with large ceiling effects (i.e., no one was choosing the last two options of a 5-point likert scale)? Is it appropriate to do?
1
vote
0answers
103 views
Best easy and straightforward way of computing McDonald's omega in a unidimensional test with 'psych' in R
I'm preparing a masterclass in Psychometrics for a course, and I would like to introduce McDonald's omega coefficient as the proper index for assessing the reliability of a scale.
For what I can see, ...
1
vote
0answers
386 views
McDonald's Omega: Assumptions, Coefficients and Interpretation
After browsing cross validated and several other sources on the web, I still cannot get a grip on McDonald's Omega as a measure of internal consistency. I have a hunch that many fellow social ...
0
votes
0answers
21 views
Fuzzy Correlation Clustering for psychometric scale/item analysis
I`m attempting to find an adequate composition of items from an psychometric questionnaire for further analysis. Unfortunately, the relatively small sample size is exceeded by a large item pool. ...
2
votes
1answer
34 views
How to proceed when collapsing ordinal items with levels that got no answer?
I have a set of 8 Likert-type items with 9 levels each. In no case do we have an item for which responses fill all possibilities and this has shown to be an issue for the SEM I'm trying to run (as ...
0
votes
0answers
46 views
Performance metric for math tasks
I have several math items which are varying in terms of difficulty. The difficulty ranges from zero to one, where zero is the most difficult one. Let's say users are solving such math tasks. I would ...
3
votes
0answers
414 views
How do you calculate interrater reliability for multiple items?
I have responses rated on 12 binary categories, treating the categories as separate items on the same measure. There are 10 raters, so Fleiss's Kappa seems appropriate. How would you compute ...
0
votes
0answers
17 views
Confirmatory Factor Analysis (CFA) concrete estimation example
I'm looking for a simple concrete example of Confirmatory Factor Analysis. What are the precise steps?
What I (think) understand so far:
Get data (random sample).
Bring in a theory along the line of
...
5
votes
3answers
100 views
Direction of correlation
There are two interdependent variables - experience and fun. Using a correlation, we can only say how fun and experience are related, and the strength of the relationship. My question is how do I ...
1
vote
1answer
32 views
How to Interpret Reflective Measurement Model?
Given $F$ is a latent construct (cause), and $Y_1$, $Y_2$, and $Y_3$ are the manifest (indicator) variables (effects) which are measured. Then, there are two groups of people (A and B). According to ...
0
votes
0answers
16 views
Instrument with Positive and Negative Affect
I am planning to use an instrument "GEQ" that contains 16 items, 4 items are grouped into the following categories:
a. Positive Affect
b. Negative Affect
c. Flow
d. Challenge
scale:
1: strongly agree;...
0
votes
1answer
85 views
Multisample SEM in R: Differences in significance level using UVI vs ULI
I'm a graduate student in psychology trying to examine the factor invariance of some psychological constructs worry (using a two-factor model) using Structural ...
-1
votes
1answer
76 views
Cause and effect Relationship
I understand that correlation can tell us the association (strength, direction) between two continuous variables. I also understand that Regression analysis can predict the behavior of the dependent ...
1
vote
0answers
58 views
Discriminant validity between traits that co-vary?
I'm hoping to do a validation study on a measure of anxiety (called the AISCs) in a stroke population, using convergent and divergent validity.
A test demonstrates convergent validity if it ...
0
votes
1answer
108 views
What is the relation between Cronbach's alpha and pair-wise correlation?
I generate multiple responses using a multivariate normal distribution with means of 5 and standard deviation of 2; I want ten responses and set the correlation between responses to be 0.5.
...
1
vote
1answer
37 views
Psychological variables: Right percentage of subjects to choose when comparing subjects depending on their level on Neuroticism (Eysenck's model)
I am trying to analyze some data regarding the relationship between Neuroticism/Stability, from Eysenck's model of personality, and ease of use (one of the dimensions of the TAM model, by Davis) when ...
3
votes
1answer
38 views
Source for claim that 2 measures that correlate at .70+ measure the same construct?
I am trying to locate a source/sources for this claim (from a reviewer):
I (and other measurement experts) believe that a correlation of .70 or
higher indicates that two constructs are very much ...
0
votes
1answer
66 views
Algorithm to score aptitude test to yield highest possible correlations with outcome
My problem concerns an aptitude test containing a set of single choice items ($x_1 x_2 .. x_n$). For each item, the participant may have selected option 1 to 5. These choices are scored dichotomously (...
1
vote
1answer
286 views
Sample size for validation of questionnaire [closed]
how to determine sample size for a questionnaire validation study in a certain population. the questionnaire has underwent psychometric validation before this.
1
vote
0answers
24 views
Survey with different response formats- how can this be validated
I was wondering whether a survey that includes questions with different response formats (i.e. Likert-type questions, yes/no, open response, multiple choice) can be validated? I've been asked to ...
2
votes
1answer
27 views
Psychometric properties on new sample with old questionnaire
When using a decades old inventory, questionnaire or factor from a questionnaire, is it important to retest the psychometric properties of this with the current sample, i.e. compute new Cronbach's ...
0
votes
1answer
54 views
What could be the next step after inconsistent exploratory factor analysis on a translated scale?
We are in the process of translating FACTG scale from English to Arabic. The scale includes 4 subscales (PWB (7 questions), SWB (7 questions), EWB (6 questions), FWB (7 questions)) .
Factor analysis ...
2
votes
0answers
41 views
Measuring latent traits: Searching for a suitable model
There is a kind of a problem related to psychometrics I keep coming across in my research again and again. Unfortunately, I failed to find appropriate statistical model to solve it. Can anybody ...
3
votes
2answers
417 views
Psychometric Meta-Analysis in R
I would like to perform a psychometric meta-analysis of stigma measurement instruments. A potential candidate for a specific analytical approach could be in my opinion the Hunter and Schmidt method: ...
1
vote
0answers
71 views
Can IRT supersede CTT nowadays?
CTT is Classical Test Theory, IRT is Item Response Theory.
IRT can handle anything CTT can. But I am curious why IRT is seldom mentioned on this site.
Can IRT supersede CTT nowadays?
1
vote
1answer
877 views
What's 'Mean item complexity'?
EFA to the data collected from my friend. R psych::faoutput as below:
...
1
vote
1answer
40 views
IRT Model for number of attempts before correct response
I have a test where respondents are able to attempt each item as many times as required until they get it correct. I am looking to fit an IRT model to data from this test in order to look at the ...
0
votes
1answer
309 views
Should I calculate Cronbach's alpha for a linear regression model?
I'm interested in modelling a trait Y using variables X1, X2, X3.
I'm using a linear regression model to predict trait Y using the independent variables and therefore want to avoid multicollinearity ...
4
votes
2answers
116 views
How to calculate Cronbach's alpha for instrument that presents every test item twice in an administration?
Background
I'm currently trying to calculate a Cronbach's alpha statistic for an instrument that has participants take every item twice during the administration. We are using a speeded mental ...
0
votes
2answers
41 views
Item Response Models with Few Measures
Are there any problems with using Item Response Theory models such as the Graded Response Model when the number of measures is small? I don't know if there are any asymptotic results that would make a ...
1
vote
1answer
62 views
Is it legitimate to average internal consistency / cronbach's alpha over multiple scales?
I'm writing a dynamical script which enables the user to select different scales to analyze their reliability. However, I want adress the overall reliability in some kind of global variable. Since the ...
1
vote
1answer
19 views
(solved: method factor) Does low cross-type covariances (compared to within-type cross-factor) imply a bad measure?
I'm not sure about the right terminology, but here goes.
I have a questionnaire measure (30 likert-items) that has been traditionally divided into five six-item factors. Each factor has two different ...
