Questions tagged [psychometrics]
Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.
287
questions
1
vote
1answer
13 views
How to intrepret one significant and one insignificant mediator and a significant direct effect in PROCESS?
I have conducted a mediation analysis with two mediators using PROCESS.
My findings are that in X -- a --> M1 -- b --> Y, a and b are significant.
However in X -- c --> M2 -- d --> Y, only d is ...
0
votes
0answers
6 views
How to dermine if a test is standardised?
What would you say, when can you assume that a test is standardised? With Standardised tests you can assume a continuous scale, which makes a lot more calculations possible. However, often times I am ...
0
votes
0answers
11 views
How to test for unidimensionality using the mirt r-package?
As it is my understanding that the ltm-package is obsolete, I'm looking for an equivalent to ltm::unidimTest() in the mirt-...
1
vote
0answers
26 views
Are there any simple pairwise comparison tests suitable for comparing emotions?
So, I'm pretty new to using statistics in a research context. I am currently writing my thesis in didactics of mathematics. In my study, a group of $10$ students are to solve two different ...
1
vote
0answers
37 views
Cronbach's Alpha with repeated missing values
I would like to use Cronbach's Alpha to check the fiability or internal consistency of a cognitive task. The task has a certain number of trials which were coded with zeros (incorrect trial) and ones (...
0
votes
1answer
26 views
Is signal detection theory a Bayesian approach?
Signal detection theory can be applied to modeling perception thresholds for defined stimuli, for example the detection of the presence of a specific signal. It is assumed both the stimulus intensity ...
0
votes
0answers
23 views
Relationship between people's insight and behavior
My main interest is to see how well people can identify the reasons behind their decisions. To measure this, I have collected 2 types of data: one set of variables that measures how important people ...
0
votes
0answers
23 views
Generalizability study of subscaled rating instrument
I have a school rating ($1$ thru $4$) instrument that consists of $9$ subscales (e.g., classroom instruction, school management etc.). Under each subscale, I have $6$ items on which rating occurs.
I ...
0
votes
0answers
15 views
Ranking Methodology
I came across an interesting method of ranking entities (products) today, and I wanted to hear some perspectives on this approach. Basically, the assessment asks a set of 30 personal value statements ...
0
votes
1answer
39 views
lmer fixed effects coefficients: which is the term of comparison for levels? [closed]
I have a lmer model with two within factors
lmer (variable ~ time * condition + (1|id), data= df)
with time having 2 levels (<...
0
votes
1answer
34 views
questionnaire item exclusion criteria
I have compiled a questionnaire consisting of items which aim to measure different aspects of elevator and staircase use.
The main idea when composing the items was that it would be the most ...
2
votes
1answer
36 views
Item sampling techniques to reduce processing time for questionnaire
I have a well-crafted draft of a questionnaire, containing 240 items. The average processing time is 20 secs per item, so the entire questionnaire would take around 1 hour 20 min to complete. I want ...
0
votes
0answers
11 views
Behavioural mapping and correlation
I want to correlate data gathered from a survey to data gathered from behaviour mapping, which is a type of systematic observation.
My point is: considering same population but different samples, can ...
0
votes
0answers
14 views
Should a factor analysis (for construct validity) be performed in a section of a test that is meant to measure knowledge in a specific subject?
As a part of a study, a survey to measure the impact of an environmental education project is being developed, and to do so, factor analysis and principal component analysis are being performed for ...
2
votes
1answer
34 views
Difference between “distractors” and “foils”
What is the difference between "distractors" and "foils" in experimental design in psychology?
0
votes
0answers
11 views
Convergent & Discriminant Validity -Same construct (measured by different meaures) in two different samples
I don't know how to go about looking at the Convergent & Discriminant Validity of the same construct measured with two different scales in two different contexts.
This is a cross-cultural ...
0
votes
0answers
8 views
Can I treat the Generalized Partial Credit Model as a scale in the Survey of Adult Skills (PIAAC)?
The Survey of Adult Skills (PIAAC) contains 250,000 rows and many columns. 6 of the columns contain the answer of questions in Problem-solving skills. Their possible values are:
The technical ...
1
vote
1answer
41 views
Correlations - multiple observations & multiple observers
I haven't found quite this question on here.
I have a data set that typically requires generalizability theory to untangle however I now face a question I believe g theory can not answer.
17 ...
0
votes
1answer
374 views
item-total correlation vs. inter-item correlation
I am assesing the reliability and validity of an 16-item multuiple choice test supposed to test language skills. Cronbach's apha is "questionable" with .65. I am having troubles interpreting the ...
2
votes
1answer
70 views
0
votes
0answers
16 views
How to test the validity of Fitt's law in R
I would like to use the Fitt's Law to test some hypothesis. Anyway, it's the first time I use it, and I have some doubts.
Then aim of my experiment is to test if in a certain situation the Fitt's law ...
0
votes
0answers
42 views
Comparing two IRT test information curves
I am fitting an IRT graded response model in R. I want to compare two measures in terms of at what levels each measure is most informative (e.g., is measure 1 informative at more extreme levels of the ...
0
votes
0answers
7 views
Comparison of discrimination tests using, Thurstonian model and its d -values
The task is to measure two discrimination test methods "triangle test and tetrad". In both methods the possibility to guess correct is (1/3).
In my study I used Chi square test to analyse data.
...
3
votes
1answer
58 views
Combining multiple response models
Within a particular instrument, is it valid to combine multiple response models for subsets of items when estimating a latent trait?
As an example, if I had some items that were conceptually suited ...
0
votes
0answers
667 views
Proper way to plot confidence interval under normal distribution curve?
I write a small program which calculates a confidence interval for a given norm value. The output is then visualized by plotting the confidence interval and the corresponding hypothetical norm value ...
0
votes
0answers
44 views
Coefficient omega: appropriate extraction of factors
I use omega_hiearchical to determine the internal consistency of a scale. If I understood it correctly, it tries to extract three lower level factors by default. But you can specify that you want to ...
2
votes
2answers
168 views
How do I calculate d' from experimental data?
In Signal Detection Theory, d' is defined by the z-scores of Hits and False Alarms:
d' = z(Hits) - z(False Alarms).
Say the task is to detect if a certain object is present in a series of pictures, ...
0
votes
0answers
18 views
Identifying problems in psychometric data set
I am analyzing the data from two questionnaires designed to measure the same trait which are highly correlated with one another and have very similar questions. My data is from one group which ...
3
votes
0answers
49 views
What is the intuition behind taking the sum of square roots, squared
In a recent publication, the authors report the following transformation when aggregating across three different scales:
Cognitive style level was used as a control variable and captured as
...
1
vote
0answers
12 views
Is it correct to a total Cronbach´s alpha analysis to a tridimensional scale?
There´s a statement that says that Cronbach´s alpha assumes unidimensionality. In that sense, is it correct to calculate a global alpha? I mean, by including all 3 dimensions, and not by separating ...
0
votes
0answers
30 views
How to interpret low fit of bifactor SEM model?
I am trying to evaluate the psychometric properties of a questionnaire instrument using bifactor SEM. A bifactor model with five group factors has the best fit, and this model has “good” RMSEA and ...
0
votes
1answer
27 views
Ceiling effect of data-IRT Rasch EFA
Can you do IRT, CFA and/or Rasch if your data has items with large ceiling effects (i.e., no one was choosing the last two options of a 5-point likert scale)? Is it appropriate to do?
1
vote
0answers
160 views
Best easy and straightforward way of computing McDonald's omega in a unidimensional test with 'psych' in R
I'm preparing a masterclass in Psychometrics for a course, and I would like to introduce McDonald's omega coefficient as the proper index for assessing the reliability of a scale.
For what I can see, ...
1
vote
0answers
746 views
McDonald's Omega: Assumptions, Coefficients and Interpretation
After browsing cross validated and several other sources on the web, I still cannot get a grip on McDonald's Omega as a measure of internal consistency. I have a hunch that many fellow social ...
2
votes
1answer
37 views
How to proceed when collapsing ordinal items with levels that got no answer?
I have a set of 8 Likert-type items with 9 levels each. In no case do we have an item for which responses fill all possibilities and this has shown to be an issue for the SEM I'm trying to run (as ...
0
votes
0answers
47 views
Performance metric for math tasks
I have several math items which are varying in terms of difficulty. The difficulty ranges from zero to one, where zero is the most difficult one. Let's say users are solving such math tasks. I would ...
3
votes
0answers
641 views
How do you calculate interrater reliability for multiple items?
I have responses rated on 12 binary categories, treating the categories as separate items on the same measure. There are 10 raters, so Fleiss's Kappa seems appropriate. How would you compute ...
0
votes
0answers
20 views
Confirmatory Factor Analysis (CFA) concrete estimation example
I'm looking for a simple concrete example of Confirmatory Factor Analysis. What are the precise steps?
What I (think) understand so far:
Get data (random sample).
Bring in a theory along the line of
...
5
votes
3answers
159 views
Direction of correlation
There are two interdependent variables - experience and fun. Using a correlation, we can only say how fun and experience are related, and the strength of the relationship. My question is how do I ...
1
vote
1answer
45 views
How to Interpret Reflective Measurement Model?
Given $F$ is a latent construct (cause), and $Y_1$, $Y_2$, and $Y_3$ are the manifest (indicator) variables (effects) which are measured. Then, there are two groups of people (A and B). According to ...
0
votes
0answers
20 views
Instrument with Positive and Negative Affect
I am planning to use an instrument "GEQ" that contains 16 items, 4 items are grouped into the following categories:
a. Positive Affect
b. Negative Affect
c. Flow
d. Challenge
scale:
1: strongly agree;...
0
votes
1answer
98 views
Multisample SEM in R: Differences in significance level using UVI vs ULI
I'm a graduate student in psychology trying to examine the factor invariance of some psychological constructs worry (using a two-factor model) using Structural ...
-1
votes
1answer
96 views
Cause and effect Relationship
I understand that correlation can tell us the association (strength, direction) between two continuous variables. I also understand that Regression analysis can predict the behavior of the dependent ...
1
vote
0answers
70 views
Discriminant validity between traits that co-vary?
I'm hoping to do a validation study on a measure of anxiety (called the AISCs) in a stroke population, using convergent and divergent validity.
A test demonstrates convergent validity if it ...
0
votes
1answer
135 views
What is the relation between Cronbach's alpha and pair-wise correlation?
I generate multiple responses using a multivariate normal distribution with means of 5 and standard deviation of 2; I want ten responses and set the correlation between responses to be 0.5.
...
1
vote
1answer
41 views
Psychological variables: Right percentage of subjects to choose when comparing subjects depending on their level on Neuroticism (Eysenck's model)
I am trying to analyze some data regarding the relationship between Neuroticism/Stability, from Eysenck's model of personality, and ease of use (one of the dimensions of the TAM model, by Davis) when ...
3
votes
1answer
38 views
Source for claim that 2 measures that correlate at .70+ measure the same construct?
I am trying to locate a source/sources for this claim (from a reviewer):
I (and other measurement experts) believe that a correlation of .70 or
higher indicates that two constructs are very much ...
0
votes
1answer
81 views
Algorithm to score aptitude test to yield highest possible correlations with outcome
My problem concerns an aptitude test containing a set of single choice items ($x_1 x_2 .. x_n$). For each item, the participant may have selected option 1 to 5. These choices are scored dichotomously (...
1
vote
1answer
345 views
Sample size for validation of questionnaire [closed]
how to determine sample size for a questionnaire validation study in a certain population. the questionnaire has underwent psychometric validation before this.
1
vote
0answers
24 views
Survey with different response formats- how can this be validated
I was wondering whether a survey that includes questions with different response formats (i.e. Likert-type questions, yes/no, open response, multiple choice) can be validated? I've been asked to ...
