Questions tagged [psychometrics]

Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

Filter by
Sorted by
Tagged with
0
votes
1answer
749 views

item-total correlation vs. inter-item correlation

I am assesing the reliability and validity of an 16-item multuiple choice test supposed to test language skills. Cronbach's apha is "questionable" with .65. I am having troubles interpreting the ...
4
votes
1answer
136 views

What's an intuitive way to explain the different types of validity?

Specifically I'm thinking of a simplified division whereby validity is divided into: Construct validity 1a. Convergent validity 1b. Discriminant validity Criterion related validity 2a. Predictive ...
7
votes
1answer
190 views

Analyzing a partially crossed design

I have a data set from a test (see below). The scoring algorithm gives each item (item_id) a score (y) that is continuous from $...
1
vote
0answers
16 views

Does it make sense to use the Spearman-Brown correction (prediction / prophecy) formula with Kendall's Tau correlation coefficient?

The Spearman-Brown prophecy formula is used to estimate the increase in reliability that is obtained when more measurements are taken (and aggregated) from the same construct. Reliability is the ...
0
votes
0answers
13 views

Trying to compute factor scores from second-order EFA analysis - non-conformable arrays error

I'm stuck at the last point of my analysis where I am trying to compute factor scores from my factor analysis. I get the following error, but I've checked that the number of rows/cols of the matrix ...
0
votes
0answers
8 views

Choosing a solution from Exploratory Factor Analysis

I have a survey with 337 responses to 31 questions (variables). All questions are on the 7 point Likert scale and I am trying to do a factor analysis. First I tried doing Confirmatory Factor Analysis ...
1
vote
0answers
22 views

How to use Cronbach's Alpha?

Is it appropriate to use Cronbach's Alpha for nominal variables? I have 20 nominal variables (two dimensions with 10 questions each) that ask about the characteristics of work performance. The ...
0
votes
0answers
17 views

Is it advisable to dichotomize an independent variable measured using a Likert scale? [duplicate]

I am working on a research project to identify the relationship between sexual esteem and sadistic behavior. My independent and dependent variable have both been measured using standard questionnaires ...
0
votes
0answers
17 views

Internal consistency: Alternative measures for unidimensional scales with a small number of items?

I use r and analyze a scale of 11 items, theoretically assuming one global factor and three group factors. Since a CFA revealed numerous correlated errors and due to the criticism of cronbachs alpha (...
0
votes
1answer
2k views

Doing CFA on a known theoretical model, but having problems with convergent and discriminant validity

I did a survey based on the UTAUT model / theory with standard questions used. The N was 150. I am trying to validate the model for my survey with structural equation modeling (via the AMOS software). ...
3
votes
0answers
19 views

How to deal with data from different centers?

I want to examine the psychometric properties (e.g. internal consistency, factorial validity) of a scale. The data (N=700) comes from 16 different locations. Central means and variances did not differ ...
0
votes
0answers
10 views

Pointers on how to proceed with my survey-method based research

I am doing a research on the effects of dominant personality and sexual self esteem on an individual's need for sexual dominance. I have decided to use the survey method for the project using two ...
1
vote
0answers
18 views

Performing a CMH test in R [closed]

I'm trying to perform a CMH test to examine differential item functioning by sex in R. I've tried cmh.test and mantelhaen.test ...
0
votes
1answer
21 views

Why can exploratory factor analysis (EFA) suggest a single factor, yet a 1-factor confirmatory factor analysis (CFA) model not fit?

I know I previously saw a thread on this topic, but I cannot locate it. So, please forgive my redundancy. I have an 8-item questionnaire administered to a (small) sample of 59 patients. The data ...
0
votes
1answer
23 views

Can I have 2 Cronbach's alphas?

I've got an overall Cronbach's alpha of -.3. The questions are reverse scored. When I calculate 5 out of the 10 questions which have high to low scoring, alpha is .8, and when I find alpha for the ...
0
votes
1answer
60 views

Correlations - multiple observations & multiple observers

I haven't found quite this question on here. I have a data set that typically requires generalizability theory to untangle however I now face a question I believe g theory can not answer. 17 ...
0
votes
1answer
28 views

Inferential interpretation of Kappa reliability

I have obtained a Fliess' Kappa interrater reliability of $37.5\%$ from $90$ randomly selected raters rating $100$ randomly selected, binary items ($Yes/No$). Question: Is the following ...
2
votes
1answer
65 views

Where I can find resources to learn how to calculate the sample size representativeness, and realiability and validity of questionnaires?

I'm totally newbie in psychometrics but starting from a research paper with full data I would like to understand: how to calculate if the sample size is representative; how to calculate reliability ...
2
votes
1answer
22 views

How to intrepret one significant and one insignificant mediator and a significant direct effect in PROCESS?

I have conducted a mediation analysis with two mediators using PROCESS. My findings are that in X -- a --> M1 -- b --> Y, a and b are significant. However in X -- c --> M2 -- d --> Y, only d is ...
0
votes
0answers
6 views

How to dermine if a test is standardised?

What would you say, when can you assume that a test is standardised? With Standardised tests you can assume a continuous scale, which makes a lot more calculations possible. However, often times I am ...
0
votes
0answers
72 views

How to test for unidimensionality using the mirt r-package?

As it is my understanding that the ltm-package is obsolete, I'm looking for an equivalent to ltm::unidimTest() in the mirt-...
1
vote
0answers
49 views

Cronbach's Alpha with repeated missing values

I would like to use Cronbach's Alpha to check the fiability or internal consistency of a cognitive task. The task has a certain number of trials which were coded with zeros (incorrect trial) and ones (...
1
vote
0answers
26 views

Are there any simple pairwise comparison tests suitable for comparing emotions?

So, I'm pretty new to using statistics in a research context. I am currently writing my thesis in didactics of mathematics. In my study, a group of $10$ students are to solve two different ...
0
votes
1answer
39 views

Is signal detection theory a Bayesian approach?

Signal detection theory can be applied to modeling perception thresholds for defined stimuli, for example the detection of the presence of a specific signal. It is assumed both the stimulus intensity ...
8
votes
5answers
15k views

Is 0 a valid value in a Likert scale?

I have carried out my pilot study on language learning motivation using a 6 point Likert scale but from 0 (strongly disagree) to 5 (very much agree). I noticed a colleague in his survey used 1 to 6. ...
24
votes
5answers
17k views

How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
0
votes
0answers
23 views

Relationship between people's insight and behavior

My main interest is to see how well people can identify the reasons behind their decisions. To measure this, I have collected 2 types of data: one set of variables that measures how important people ...
0
votes
0answers
23 views

Generalizability study of subscaled rating instrument

I have a school rating ($1$ thru $4$) instrument that consists of $9$ subscales (e.g., classroom instruction, school management etc.). Under each subscale, I have $6$ items on which rating occurs. I ...
3
votes
5answers
580 views

How can there be a standard deviation of true scores?

Dudek (1979) describes the following formula as "the standard deviation of true scores when the observed score is held constant" (p. 337): $SE_{est} = SD \sqrt{(reliability × (1 - reliability))}$ ...
8
votes
2answers
189 views

A person repeatedly selects the two most similar items out of three. How to model/estimate a perceptual distance between the items?

A person is given three items, say pictures of faces, and is asked to pick out which two of the three faces are the most similar. This is repeated a large number of times with different combinations ...
0
votes
0answers
15 views

Ranking Methodology

I came across an interesting method of ranking entities (products) today, and I wanted to hear some perspectives on this approach. Basically, the assessment asks a set of 30 personal value statements ...
0
votes
1answer
35 views

questionnaire item exclusion criteria

I have compiled a questionnaire consisting of items which aim to measure different aspects of elevator and staircase use. The main idea when composing the items was that it would be the most ...
0
votes
1answer
63 views

lmer fixed effects coefficients: which is the term of comparison for levels? [closed]

I have a lmer model with two within factors lmer (variable ~ time * condition + (1|id), data= df) with time having 2 levels (<...
3
votes
1answer
491 views

Cronbach's alpha vs. Average Item-Sum correlation?

I have heard that Cronbach's alpha is used to measure reliability of a test (to check if a set of items/question are measuring the same construct). How does the Cronbach's alpha compare with the ...
2
votes
1answer
38 views

Item sampling techniques to reduce processing time for questionnaire

I have a well-crafted draft of a questionnaire, containing 240 items. The average processing time is 20 secs per item, so the entire questionnaire would take around 1 hour 20 min to complete. I want ...
3
votes
1answer
1k views

Why don't we square the reliability correlation when calculating reliable variance?

This question relates to situations where we apply some test to the same people at Time 1 and Time 2, and then calculate the correlation between scores at Time 1 and Time 2. I am reading the book ...
3
votes
1answer
43 views

Difference between “distractors” and “foils”

What is the difference between "distractors" and "foils" in experimental design in psychology?
2
votes
2answers
888 views

Usual practices on ordinal data in psychometric tests (z-score, percentiles, t-test)

I want to build a standardised score on ordinal data. Data comes from several scales from a personality test of summed up rating items. It seems that ordinal data in psychology is often treated like ...
8
votes
1answer
678 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
-1
votes
1answer
847 views

What is a design matrix? How do I build a design matrix for Rasch analysis?

I am trying to do an LRSM Rasch analysis of 72-item data. This data has 67 binary items and 5 ternary items. I need to create a "design matrix" in order to analyze this data. I do not really know what ...
2
votes
1answer
76 views
3
votes
1answer
61 views

Combining multiple response models

Within a particular instrument, is it valid to combine multiple response models for subsets of items when estimating a latent trait? As an example, if I had some items that were conceptually suited ...
0
votes
1answer
247 views

Detecting Differential Item Functioning (DIF) using ordered logistic regression

I am in a situation in which I want to investigate whether test items of a questionnaire appear to have DIF. The items are polytomous and ranges from 1 to 5, with 5 being the most positive answer. I'...
0
votes
0answers
955 views

Proper way to plot confidence interval under normal distribution curve?

I write a small program which calculates a confidence interval for a given norm value. The output is then visualized by plotting the confidence interval and the corresponding hypothetical norm value ...
10
votes
1answer
18k views

Intraclass correlation coefficient vs. F-test (one-way ANOVA)?

I'm a little confused regarding the intraclass correlation coefficient and one-way ANOVA. As I understand it, both tell you how similar observations within a group are, relative to observations in ...
3
votes
2answers
2k views

Acceptable values for discrimination parameter - 2PL IRT Model

What is the acceptable range for discrimination parameters with the two-parameter IRT model? I have information from Baker (2001) but would love additional information. I have heard discrimination ...
2
votes
2answers
237 views

How do I calculate d' from experimental data?

In Signal Detection Theory, d' is defined by the z-scores of Hits and False Alarms: d' = z(Hits) - z(False Alarms). Say the task is to detect if a certain object is present in a series of pictures, ...
3
votes
0answers
129 views

What exactly is psychometric modelling?

My current definition of psychometric modelling is as follows: The construction, via statistical tools such as the Rasch or DINA model, of a simplified reprensentation of the operations that lead ...
9
votes
4answers
419 views

How should I approach this binary prediction problem?

I've got a dataset with the following format. There's a binary outcome cancer/no cancer. Every doctor in the dataset has seen every patient and given an independent judgment on whether the patient ...
3
votes
0answers
59 views

What is the intuition behind taking the sum of square roots, squared

In a recent publication, the authors report the following transformation when aggregating across three different scales: Cognitive style level was used as a control variable and captured as ...

1
2 3 4 5 6