Questions tagged [psychometrics]
Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.
280
questions
0
votes
0answers
12 views
Ranking Methodology
I came across an interesting method of ranking entities (products) today, and I wanted to hear some perspectives on this approach. Basically, the assessment asks a set of 30 personal value statements ...
0
votes
1answer
36 views
lmer fixed effects coefficients: which is the term of comparison for levels? [closed]
I have a lmer model with two within factors
lmer (variable ~ time * condition + (1|id), data= df)
with time having 2 levels (<...
0
votes
1answer
30 views
questionnaire item exclusion criteria
I have compiled a questionnaire consisting of items which aim to measure different aspects of elevator and staircase use.
The main idea when composing the items was that it would be the most ...
2
votes
1answer
35 views
Item sampling techniques to reduce processing time for questionnaire
I have a well-crafted draft of a questionnaire, containing 240 items. The average processing time is 20 secs per item, so the entire questionnaire would take around 1 hour 20 min to complete. I want ...
0
votes
0answers
9 views
Behavioural mapping and correlation
I want to correlate data gathered from a survey to data gathered from behaviour mapping, which is a type of systematic observation.
My point is: considering same population but different samples, can ...
0
votes
0answers
8 views
Should a factor analysis (for construct validity) be performed in a section of a test that is meant to measure knowledge in a specific subject?
As a part of a study, a survey to measure the impact of an environmental education project is being developed, and to do so, factor analysis and principal component analysis are being performed for ...
2
votes
1answer
26 views
Difference between “distractors” and “foils”
What is the difference between "distractors" and "foils" in experimental design in psychology?
0
votes
0answers
7 views
Convergent & Discriminant Validity -Same construct (measured by different meaures) in two different samples
I don't know how to go about looking at the Convergent & Discriminant Validity of the same construct measured with two different scales in two different contexts.
This is a cross-cultural ...
0
votes
0answers
7 views
Can I treat the Generalized Partial Credit Model as a scale in the Survey of Adult Skills (PIAAC)?
The Survey of Adult Skills (PIAAC) contains 250,000 rows and many columns. 6 of the columns contain the answer of questions in Problem-solving skills. Their possible values are:
The technical ...
1
vote
1answer
34 views
Correlations - multiple observations & multiple observers
I haven't found quite this question on here.
I have a data set that typically requires generalizability theory to untangle however I now face a question I believe g theory can not answer.
17 ...
0
votes
1answer
92 views
item-total correlation vs. inter-item correlation
I am assesing the reliability and validity of an 16-item multuiple choice test supposed to test language skills. Cronbach's apha is "questionable" with .65. I am having troubles interpreting the ...
2
votes
1answer
68 views
0
votes
0answers
13 views
How to test the validity of Fitt's law in R
I would like to use the Fitt's Law to test some hypothesis. Anyway, it's the first time I use it, and I have some doubts.
Then aim of my experiment is to test if in a certain situation the Fitt's law ...
0
votes
0answers
23 views
Comparing two IRT test information curves
I am fitting an IRT graded response model in R. I want to compare two measures in terms of at what levels each measure is most informative (e.g., is measure 1 informative at more extreme levels of the ...
0
votes
0answers
6 views
Comparison of discrimination tests using, Thurstonian model and its d -values
The task is to measure two discrimination test methods "triangle test and tetrad". In both methods the possibility to guess correct is (1/3).
In my study I used Chi square test to analyse data.
...
3
votes
1answer
52 views
Combining multiple response models
Within a particular instrument, is it valid to combine multiple response models for subsets of items when estimating a latent trait?
As an example, if I had some items that were conceptually suited ...
0
votes
0answers
414 views
Proper way to plot confidence interval under normal distribution curve?
I write a small program which calculates a confidence interval for a given norm value. The output is then visualized by plotting the confidence interval and the corresponding hypothetical norm value ...
0
votes
0answers
38 views
Coefficient omega: appropriate extraction of factors
I use omega_hiearchical to determine the internal consistency of a scale. If I understood it correctly, it tries to extract three lower level factors by default. But you can specify that you want to ...
2
votes
2answers
127 views
How do I calculate d' from experimental data?
In Signal Detection Theory, d' is defined by the z-scores of Hits and False Alarms:
d' = z(Hits) - z(False Alarms).
Say the task is to detect if a certain object is present in a series of pictures, ...
0
votes
0answers
16 views
Identifying problems in psychometric data set
I am analyzing the data from two questionnaires designed to measure the same trait which are highly correlated with one another and have very similar questions. My data is from one group which ...
3
votes
0answers
47 views
What is the intuition behind taking the sum of square roots, squared
In a recent publication, the authors report the following transformation when aggregating across three different scales:
Cognitive style level was used as a control variable and captured as
...
1
vote
0answers
12 views
Is it correct to a total Cronbach´s alpha analysis to a tridimensional scale?
There´s a statement that says that Cronbach´s alpha assumes unidimensionality. In that sense, is it correct to calculate a global alpha? I mean, by including all 3 dimensions, and not by separating ...
0
votes
0answers
22 views
How to interpret low fit of bifactor SEM model?
I am trying to evaluate the psychometric properties of a questionnaire instrument using bifactor SEM. A bifactor model with five group factors has the best fit, and this model has “good” RMSEA and ...
0
votes
1answer
25 views
Ceiling effect of data-IRT Rasch EFA
Can you do IRT, CFA and/or Rasch if your data has items with large ceiling effects (i.e., no one was choosing the last two options of a 5-point likert scale)? Is it appropriate to do?
1
vote
0answers
122 views
Best easy and straightforward way of computing McDonald's omega in a unidimensional test with 'psych' in R
I'm preparing a masterclass in Psychometrics for a course, and I would like to introduce McDonald's omega coefficient as the proper index for assessing the reliability of a scale.
For what I can see, ...
1
vote
0answers
502 views
McDonald's Omega: Assumptions, Coefficients and Interpretation
After browsing cross validated and several other sources on the web, I still cannot get a grip on McDonald's Omega as a measure of internal consistency. I have a hunch that many fellow social ...
0
votes
0answers
23 views
Fuzzy Correlation Clustering for psychometric scale/item analysis
I`m attempting to find an adequate composition of items from an psychometric questionnaire for further analysis. Unfortunately, the relatively small sample size is exceeded by a large item pool. ...
2
votes
1answer
36 views
How to proceed when collapsing ordinal items with levels that got no answer?
I have a set of 8 Likert-type items with 9 levels each. In no case do we have an item for which responses fill all possibilities and this has shown to be an issue for the SEM I'm trying to run (as ...
0
votes
0answers
46 views
Performance metric for math tasks
I have several math items which are varying in terms of difficulty. The difficulty ranges from zero to one, where zero is the most difficult one. Let's say users are solving such math tasks. I would ...
3
votes
0answers
530 views
How do you calculate interrater reliability for multiple items?
I have responses rated on 12 binary categories, treating the categories as separate items on the same measure. There are 10 raters, so Fleiss's Kappa seems appropriate. How would you compute ...
0
votes
0answers
17 views
Confirmatory Factor Analysis (CFA) concrete estimation example
I'm looking for a simple concrete example of Confirmatory Factor Analysis. What are the precise steps?
What I (think) understand so far:
Get data (random sample).
Bring in a theory along the line of
...
5
votes
3answers
111 views
Direction of correlation
There are two interdependent variables - experience and fun. Using a correlation, we can only say how fun and experience are related, and the strength of the relationship. My question is how do I ...
1
vote
1answer
40 views
How to Interpret Reflective Measurement Model?
Given $F$ is a latent construct (cause), and $Y_1$, $Y_2$, and $Y_3$ are the manifest (indicator) variables (effects) which are measured. Then, there are two groups of people (A and B). According to ...
0
votes
0answers
19 views
Instrument with Positive and Negative Affect
I am planning to use an instrument "GEQ" that contains 16 items, 4 items are grouped into the following categories:
a. Positive Affect
b. Negative Affect
c. Flow
d. Challenge
scale:
1: strongly agree;...
0
votes
1answer
87 views
Multisample SEM in R: Differences in significance level using UVI vs ULI
I'm a graduate student in psychology trying to examine the factor invariance of some psychological constructs worry (using a two-factor model) using Structural ...
-1
votes
1answer
88 views
Cause and effect Relationship
I understand that correlation can tell us the association (strength, direction) between two continuous variables. I also understand that Regression analysis can predict the behavior of the dependent ...
1
vote
0answers
62 views
Discriminant validity between traits that co-vary?
I'm hoping to do a validation study on a measure of anxiety (called the AISCs) in a stroke population, using convergent and divergent validity.
A test demonstrates convergent validity if it ...
0
votes
1answer
120 views
What is the relation between Cronbach's alpha and pair-wise correlation?
I generate multiple responses using a multivariate normal distribution with means of 5 and standard deviation of 2; I want ten responses and set the correlation between responses to be 0.5.
...
1
vote
1answer
40 views
Psychological variables: Right percentage of subjects to choose when comparing subjects depending on their level on Neuroticism (Eysenck's model)
I am trying to analyze some data regarding the relationship between Neuroticism/Stability, from Eysenck's model of personality, and ease of use (one of the dimensions of the TAM model, by Davis) when ...
3
votes
1answer
38 views
Source for claim that 2 measures that correlate at .70+ measure the same construct?
I am trying to locate a source/sources for this claim (from a reviewer):
I (and other measurement experts) believe that a correlation of .70 or
higher indicates that two constructs are very much ...
0
votes
1answer
77 views
Algorithm to score aptitude test to yield highest possible correlations with outcome
My problem concerns an aptitude test containing a set of single choice items ($x_1 x_2 .. x_n$). For each item, the participant may have selected option 1 to 5. These choices are scored dichotomously (...
1
vote
1answer
304 views
Sample size for validation of questionnaire [closed]
how to determine sample size for a questionnaire validation study in a certain population. the questionnaire has underwent psychometric validation before this.
1
vote
0answers
24 views
Survey with different response formats- how can this be validated
I was wondering whether a survey that includes questions with different response formats (i.e. Likert-type questions, yes/no, open response, multiple choice) can be validated? I've been asked to ...
2
votes
1answer
28 views
Psychometric properties on new sample with old questionnaire
When using a decades old inventory, questionnaire or factor from a questionnaire, is it important to retest the psychometric properties of this with the current sample, i.e. compute new Cronbach's ...
0
votes
1answer
64 views
What could be the next step after inconsistent exploratory factor analysis on a translated scale?
We are in the process of translating FACTG scale from English to Arabic. The scale includes 4 subscales (PWB (7 questions), SWB (7 questions), EWB (6 questions), FWB (7 questions)) .
Factor analysis ...
2
votes
0answers
41 views
Measuring latent traits: Searching for a suitable model
There is a kind of a problem related to psychometrics I keep coming across in my research again and again. Unfortunately, I failed to find appropriate statistical model to solve it. Can anybody ...
3
votes
2answers
463 views
Psychometric Meta-Analysis in R
I would like to perform a psychometric meta-analysis of stigma measurement instruments. A potential candidate for a specific analytical approach could be in my opinion the Hunter and Schmidt method: ...
1
vote
0answers
73 views
Can IRT supersede CTT nowadays?
CTT is Classical Test Theory, IRT is Item Response Theory.
IRT can handle anything CTT can. But I am curious why IRT is seldom mentioned on this site.
Can IRT supersede CTT nowadays?
1
vote
1answer
1k views
What's 'Mean item complexity'?
EFA to the data collected from my friend. R psych::faoutput as below:
...
1
vote
1answer
41 views
IRT Model for number of attempts before correct response
I have a test where respondents are able to attempt each item as many times as required until they get it correct. I am looking to fit an IRT model to data from this test in order to look at the ...
