Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

1
vote
2answers
61 views
+50

Performance metric based on five-star ratings

Although I think my question might be a specific case of a generic problem, I'll include some background info that might be relevant. There is a sort of concierge/delivery service I know that uses a ...
2
votes
2answers
340 views
0
votes
0answers
17 views

Consequences of violating assumptions about errors in IRT/Rasch based models

In the sorts of measurement models to which one might apply IRT models, including Rasch models, there is not thought to be an error associated with the latent variable. Rather, the latent variable is ...
3
votes
2answers
30 views

What is the best method to assess how “unique” a covariate is?

Essentially, I have about 20 variables (all numerical scores) for different personality measures. I want to measure which is the most unique, ie [edit: the least predicted by the other variables.] ...
0
votes
0answers
16 views

Multidimensional constructs and interrelationship between the subdimensions

Is anyone aware of the scientific term one would use when we look at the relationship between several first order constructs that together is used to form a formative aggregated multidimensional ...
3
votes
1answer
265 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
0
votes
0answers
10 views

Unidimensional but not interrelated?

Internal consistency is a necessary but not sufficient condition for homogeneity or unidimensionality - Clark & Watson (1995) In practice, it is possible to have unidimensionality between items ...
0
votes
0answers
6 views

what is the difference between generalizability theory and power analysis

Genearalizability theory, especially the decision study(D study), sounds very similar to Power analysis. A D study uses the variance components from previous variance composition of the measurement ...
4
votes
1answer
934 views

What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
4
votes
1answer
323 views

Omega H asymptotic

In the omega function in the psych package for R, we have several outputs including Hierarchical omega, Asymptotic omega and Total omega. I was reading about the outputs, but I find nothing about ...
1
vote
1answer
731 views

Doing CFA on a known theoretical model, but having problems with convergent and discriminant validity

I did a survey based on the UTAUT model / theory with standard questions used. The N was 150. I am trying to validate the model for my survey with structural equation modeling (via the AMOS software). ...
1
vote
0answers
10 views

(Psychology research) Is it appropriate or coherent to standardize these variables?

I am doing a project where I'm analyzing various lexical dimensions of speech from folktales from 10 countries. And the program, LIWC, returns a value for each category with that value representing ...
2
votes
2answers
90 views

Acceptable values for discrimination parameter - 2PL IRT Model

What is the acceptable range for discrimination parameters with the two-parameter IRT model? I have information from Baker (2001) but would love additional information. I have heard discrimination ...
0
votes
0answers
7 views

How to test if the difference between two logit-function-fits is significant or not?

I am considering a binomial experiment where each trial can only have two possible outputs (0,1). I vary the value of a predictor (experiment parameter) and repeat the experiment several times for ...
0
votes
0answers
14 views

How can I account for multiple standardized tests being used during a longitudinal study?

I am currently conducting a longitudinal study of the growth of student test scores over time, and many of the districts in my study have switched to a new state assessment. The new assessment and the ...
2
votes
0answers
39 views

Appropriateness of Rasch for formative measurement models

I've spent several months thinking about the issue of whether or not it is appropriate to apply Rasch models to formative measurement models, and I'm looking to see whether anybody else has considered ...
1
vote
0answers
18 views

intraclass correlation (ICC) for level of comprehension of individual items by different raters

I'm currently doing a translation and adaptation of a questionnaire composed of 36 items. After finishing the translation, synthesis and back-translation steps, 40 people randomly selected from a ...
0
votes
0answers
36 views

Principal Component Analysis: Scale reliability and validity

I'm using a popular psychometric scale, the Domain Specific Risk Taking (DoSpeRT) scale, that contains 30 items that measure an individual's risk taking capacity across five different dimensions: ...
0
votes
1answer
99 views

Average inter-item correlations as internal consistency indicator

In literature, the average inter-item correlations in the range of 0.15 to 0.50 is considered as an indicator for an acceptable level of consistency. Why do the high average inter-item correlations ...
0
votes
0answers
11 views

“*usefulness*” is a bivariate property used in Regression and Anova. Has a generalization (trivariate) analogon been discussed?

Just for selfstudy/exercising of algorithms I looked at the computation of the "usefulness"-measure in multiple regression, which means the part of variance which one independent item contributes to ...
1
vote
1answer
561 views

Calculate Cronbach's alpha for polytomous items in R

Does anyone know whether there is a function which can calculate Cronbach's alpha for polytomous items with different categories in R?
2
votes
1answer
34 views

Cronbach's alpha vs. Average Item-Sum correlation?

I have heard that Cronbach's alpha is used to measure reliability of a test (to check if a set of items/question are measuring the same construct). How does the Cronbach's alpha compare with the ...
4
votes
1answer
758 views

What is the correct use of Cronbach's alpha?

I am trying to use Cronbach's alpha in order to see if some variables are measuring the "same", but I have some doubts about the correct use: Can I compute alpha for numerical, ordinal or ...
3
votes
0answers
40 views

Does weighted factor analysis exist?

I would like to perform factor analysis on a set of responses to a psychological questionnaire. Is there some method that allows me to "weight" each questionnaire item by how important I believe it is ...
5
votes
1answer
6k views

Cronbach's alpha negative result

What should I do if I am getting negative results for Cronbach's alpha? Is there anyway to adjust it?
6
votes
1answer
148 views

What is Cronbach's Alpha intuitively?

I'm trying to understand Cronbach's Alpha intuitively. What is the general idea behind this construct? What properties were they trying to ensure that it had?
-5
votes
1answer
191 views

Did Normal distribution theory originate from psychometric analysis especially reliability and validity analysis? [closed]

The basic statistical literature does not talk much about the exact background and history of the normal distribution theory which provides us wth a base for traditional(descriptive analysis or ...
2
votes
1answer
587 views

Cronbach's Alpha on true and false data?

I have a 10 item knowledge scale (true/false). Can I do Cronbach alpha test?
7
votes
3answers
14k views

Is it acceptable to use Cronbach's alpha to assess reliability of questionnaire composed of categorical and conditional items?

Background: I have data on a questionnaire made up of categorical factual items. Some of them are binary and others have more than two categories. For example: ...
6
votes
1answer
545 views

How high is too high with Cronbach's alpha?

Someone in my organization just sent me a scale they developed (modified items from five separate scales used in published research) to measure an employee's readiness for change. They tested the ...
12
votes
2answers
11k views

Where do the descriptors for Cronbach's alpha values come from (e.g., poor, excellent)?

It seems fairly common to describe Cronbach's alpha values as follows: α ≥ 0.9 Excellent 0.7 ≤ α < 0.9 Good 0.6 ≤ α < 0.7 Acceptable 0.5 ≤ α < 0.6 Poor α < 0.5 Unacceptable Where ...
13
votes
2answers
4k views

Assessing reliability of a questionnaire: dimensionality, problematic items, and whether to use alpha, lambda6 or some other index?

I am analyzing scores given by participants attending an experiment. I want to estimate the reliability of my questionnaire which is composed of 6 items aimed at estimating the attitude of the ...
6
votes
2answers
1k views

Advice on scientifically sound scale construction

I've been given a set of 20 Likert-items (ranging from 1-5, sample size n = 299) within the field of organizational research. The items are intended to measure a latent concept which is ...
2
votes
1answer
197 views

Two treatments - How do I compute reliability estimates like Cronbach's $\alpha$?

I randomized my survey, and now I have my data on SPSS (and all cleaned up) but when trying to run my Cronbach's $\alpha$ I have a problem. Because I have two treatments (sexualised and normal) all ...
7
votes
2answers
2k views

Good internal factor structure but poor Cronbach's $\alpha$?

I am running a CFA and getting good fit indices (CFI = .99, RMSEA = .01) for a uni-dimensional scale. However, when I test for internal consistency, I get poor Cronbach's $\alpha$s ($\alpha = .6$). I ...
3
votes
1answer
348 views

Why change items which don't change Cronbach Alpha?

I am doing peer review on a paper submission for a journal, and I am somewhat out of my depth. The authors are developing a psychometric scale. They did a pilot study with 45 items. They measured ...
2
votes
0answers
319 views

Maximizing Cronbach alpha to select the number of variables that define a given construct

I can't seem to find this anywhere on the internet or in my books. I'm putting together a construct for an underlying variable. Is it acceptable to have a construct with two variables if that ...
0
votes
0answers
27 views

Scaled vs. unscaled Chi-square statistic in CFA / GRM fit evaluation

I generally want to assess the fit of a CFA model with ordered categorical indicators (the graded response model, GRM). There are two versions of this model, the normal-ogive GRM (ordered CFA with ...
1
vote
1answer
14 views

Randomization scheme for rating images and assessing inter-rater reliability

Apologies for the basic question, but I'm new to the area and I'm working on a research project on rating images, and assessing the reliability of these ratings, with the following structure: There ...
0
votes
1answer
32 views

What is a design matrix? How do I build a design matrix for Rasch analysis?

I am trying to do an LRSM Rasch analysis of 72-item data. This data has 67 binary items and 5 ternary items. I need to create a "design matrix" in order to analyze this data. I do not really know what ...
4
votes
4answers
402 views

Are discrimination parameters in two-parameter IRT models only specific to items?

Discrimination parameters in two-parameter model from IRT are usually considered item parameters. But I've come to doubt it. Think about psychophysics; for example, detecting luminance. I don't ...
2
votes
2answers
539 views

How is Item Response Theory (IRT) called for continuous response?

I would like to model my problem using something similar to Item Response Theory - but my responses are not binary - they are continuous in [0;1]. How are these models/the research field called?
0
votes
0answers
55 views

convergence warning in glmer(),

Ok I have a dataset which is looking at choice behavior of rats in an olfactory decision-making task. In this task the rat has to sniff at a center odor port and depending on which odor stimulus it ...
1
vote
1answer
71 views

What does 'downward pressure on sample size' mean?

In this article, there is this phrase 'downward pressure on sample size'. This is the paragraph: no matter the approach to collecting this new data, when attempting to ...
2
votes
1answer
25 views

Correspondence between two different depression scales?

I have become involved in a project where psychiatrists are studying the correspondence between two scales measuring the level of depression among a specific patient group. One of the scales is for ...
0
votes
0answers
36 views

Test-retest reliability

I have come across following table in a publication, where r is correlation coefficient. I could not find detailed procedures for test-retest reliability, is it any different from paired t-test ? ...
1
vote
0answers
37 views

How can measure be unreliable but valid?

Some previous research found evidence for the predictive validity of a measure. I've done some work and found the measure to be unreliable. Since reliability provides an upper limit for predictive ...
0
votes
0answers
8 views

How to form multi-item scales after doing a factor analysis

I am doing a content analysis research for bringing out news frames, and decided use Semetko and Valkenburg (2000) method, which consists in developing a set of questions that will help measure ...
0
votes
0answers
27 views

How to equate the cut scores

I have two different tests with two different cut scores. For example a test has a cut score of 50 and the other test has a cut score 55. I want to equate the cut scores for two test. I have ...
4
votes
2answers
119 views

The valid variance components to include in a generalizability theory model (anova) involving nesting? Are these correct interactions?

My question is about generalizability theory. A study by Bergeron, Floyd, McCormack, and Farmer (2008) is only adding to this confusion. Basically, in G-theory you want the first model you run to ...