Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

0
votes
0answers
16 views

Multidimensional constructs and interrelationship between the subdimensions

Is anyone aware of the scientific term one would use when we look at the relationship between several first order constructs that together is used to form a formative aggregated multidimensional ...
1
vote
2answers
67 views

Performance metric based on five-star ratings

Although I think my question might be a specific case of a generic problem, I'll include some background info that might be relevant. There is a sort of concierge/delivery service I know that uses a ...
0
votes
0answers
17 views

Consequences of violating assumptions about errors in IRT/Rasch based models

In the sorts of measurement models to which one might apply IRT models, including Rasch models, there is not thought to be an error associated with the latent variable. Rather, the latent variable is ...
0
votes
0answers
10 views

Unidimensional but not interrelated?

Internal consistency is a necessary but not sufficient condition for homogeneity or unidimensionality - Clark & Watson (1995) In practice, it is possible to have unidimensionality between items ...
3
votes
2answers
30 views

What is the best method to assess how “unique” a covariate is?

Essentially, I have about 20 variables (all numerical scores) for different personality measures. I want to measure which is the most unique, ie [edit: the least predicted by the other variables.] ...
0
votes
0answers
6 views

what is the difference between generalizability theory and power analysis

Genearalizability theory, especially the decision study(D study), sounds very similar to Power analysis. A D study uses the variance components from previous variance composition of the measurement ...
1
vote
0answers
10 views

(Psychology research) Is it appropriate or coherent to standardize these variables?

I am doing a project where I'm analyzing various lexical dimensions of speech from folktales from 10 countries. And the program, LIWC, returns a value for each category with that value representing ...
0
votes
0answers
7 views

How to test if the difference between two logit-function-fits is significant or not?

I am considering a binomial experiment where each trial can only have two possible outputs (0,1). I vary the value of a predictor (experiment parameter) and repeat the experiment several times for ...
0
votes
0answers
14 views

How can I account for multiple standardized tests being used during a longitudinal study?

I am currently conducting a longitudinal study of the growth of student test scores over time, and many of the districts in my study have switched to a new state assessment. The new assessment and the ...
2
votes
0answers
39 views

Appropriateness of Rasch for formative measurement models

I've spent several months thinking about the issue of whether or not it is appropriate to apply Rasch models to formative measurement models, and I'm looking to see whether anybody else has considered ...
1
vote
0answers
18 views

intraclass correlation (ICC) for level of comprehension of individual items by different raters

I'm currently doing a translation and adaptation of a questionnaire composed of 36 items. After finishing the translation, synthesis and back-translation steps, 40 people randomly selected from a ...
0
votes
0answers
36 views

Principal Component Analysis: Scale reliability and validity

I'm using a popular psychometric scale, the Domain Specific Risk Taking (DoSpeRT) scale, that contains 30 items that measure an individual's risk taking capacity across five different dimensions: ...
0
votes
0answers
11 views

“*usefulness*” is a bivariate property used in Regression and Anova. Has a generalization (trivariate) analogon been discussed?

Just for selfstudy/exercising of algorithms I looked at the computation of the "usefulness"-measure in multiple regression, which means the part of variance which one independent item contributes to ...
3
votes
0answers
42 views

Does weighted factor analysis exist?

I would like to perform factor analysis on a set of responses to a psychological questionnaire. Is there some method that allows me to "weight" each questionnaire item by how important I believe it is ...
2
votes
1answer
35 views

Cronbach's alpha vs. Average Item-Sum correlation?

I have heard that Cronbach's alpha is used to measure reliability of a test (to check if a set of items/question are measuring the same construct). How does the Cronbach's alpha compare with the ...
0
votes
0answers
27 views

Scaled vs. unscaled Chi-square statistic in CFA / GRM fit evaluation

I generally want to assess the fit of a CFA model with ordered categorical indicators (the graded response model, GRM). There are two versions of this model, the normal-ogive GRM (ordered CFA with ...
0
votes
1answer
32 views

What is a design matrix? How do I build a design matrix for Rasch analysis?

I am trying to do an LRSM Rasch analysis of 72-item data. This data has 67 binary items and 5 ternary items. I need to create a "design matrix" in order to analyze this data. I do not really know what ...
1
vote
1answer
15 views

Randomization scheme for rating images and assessing inter-rater reliability

Apologies for the basic question, but I'm new to the area and I'm working on a research project on rating images, and assessing the reliability of these ratings, with the following structure: There ...
0
votes
0answers
55 views

convergence warning in glmer(),

Ok I have a dataset which is looking at choice behavior of rats in an olfactory decision-making task. In this task the rat has to sniff at a center odor port and depending on which odor stimulus it ...
2
votes
1answer
25 views

Correspondence between two different depression scales?

I have become involved in a project where psychiatrists are studying the correspondence between two scales measuring the level of depression among a specific patient group. One of the scales is for ...
0
votes
1answer
102 views

Average inter-item correlations as internal consistency indicator

In literature, the average inter-item correlations in the range of 0.15 to 0.50 is considered as an indicator for an acceptable level of consistency. Why do the high average inter-item correlations ...
1
vote
0answers
37 views

How can measure be unreliable but valid?

Some previous research found evidence for the predictive validity of a measure. I've done some work and found the measure to be unreliable. Since reliability provides an upper limit for predictive ...
0
votes
0answers
8 views

How to form multi-item scales after doing a factor analysis

I am doing a content analysis research for bringing out news frames, and decided use Semetko and Valkenburg (2000) method, which consists in developing a set of questions that will help measure ...
0
votes
0answers
27 views

How to equate the cut scores

I have two different tests with two different cut scores. For example a test has a cut score of 50 and the other test has a cut score 55. I want to equate the cut scores for two test. I have ...
0
votes
0answers
14 views

comparing SEM in Z and T scores

I have a function that gives me the SEM of my test for different thetas. This - I believe - is a SEM of Z scores. ...
1
vote
0answers
42 views

how to remove outliers prior to multiple imputation

A colleague came to me with the following problem. She has a complex, multivariate data set, in which respondents completed a number of measures with anywhere from 6 to 30 Likert type items for each ...
0
votes
0answers
104 views

How to calculate discrimination index and difficulty index of short answer test items?

How to calculate discrimination index and difficulty index of short answer test items? Should I use t test or the formula upper group score - lower group score divided by number of students in a group?...
1
vote
0answers
22 views

Select and weigh questions on a reading assessment

I am a phd student in computer science and as such I am the goto guy for anything "mathy" in the cross discipline research group to which I belong. I have recently been given an assignment at work I'm ...
1
vote
0answers
11 views

Inter-raters reliability for sentiment analysis lexicon

I have a list of words (more than 2000) which were rated by 4 raters for sentiments like anger (scale from 1 to 5). I run an alpha test on SPSS which gives me a coefficient of >8 which shows a good ...
2
votes
2answers
344 views

What is the two alternative forced choice paradigm

Is a two alternative forced choice paradigm (2AFC) an experimental design?
1
vote
1answer
46 views

raising the probability of correct answers of nextItem (catR)

Using the catR package I am trying to create a nextItem call that respondents would answer correctly about 65% of the time (even if that means the method is not as efficient as it could be). Children ...
0
votes
0answers
36 views

Test-retest reliability

I have come across following table in a publication, where r is correlation coefficient. I could not find detailed procedures for test-retest reliability, is it any different from paired t-test ? ...
0
votes
0answers
26 views

Calculate within and between group correlation

MY DATA: A surveys which contains 2 different questionnaires on the same subject which were taken by n=800 patients (i.e., timepoint) (#1 questionnaire: 5 questions which reach a total score of 0-42, ...
0
votes
0answers
28 views

Statistical Method for Normalization of Data

I collected data using a pre-test instrument to capture baseline emotions (two dimensions: valence and arousal) of participants. The two dimensions in the questionnaire range from calm to excited (...
3
votes
0answers
27 views

Comparing confirmatory factor analysis models with different number of observed variables

I have a 14 item scale that is hypothesized to measure self-esteem. The first model I specify uses all 14 items and predicts one latent factor (i.e., self-esteem). I would then like to specify a ...
5
votes
1answer
385 views

What is the difference between formative and reflective measurement models?

Can anyone provide examples as to why choose one over the other? Are they calculated diferently?
1
vote
0answers
20 views

CFA model with systematic temporal correlation between measures

I'm looking to fit CFA models to data in the case where measures have systematic temporal correlations between them. I'm familiar with autoregressive approaches, such as those described on Dave Kenny'...
3
votes
1answer
66 views

Non-Significant difference between individual items, but significant difference between overall scores

Say that we have a scale measuring Healthiness of Diet, and want to see if two populations have different levels of Healthiness of Diet. If we find that the difference between the two samples at the ...
9
votes
3answers
253 views

Is it always better to extract more factors when they exist?

Unlike principal components analysis, the solutions to factor analysis models are not necessarily nested. That is, the loadings (for example) for the first factor won't necessarily be identical when ...
0
votes
1answer
83 views

Detecting Differential Item Functioning (DIF) using ordered logistic regression

I am in a situation in which I want to investigate whether test items of a questionnaire appear to have DIF. The items are polytomous and ranges from 1 to 5, with 5 being the most positive answer. I'...
2
votes
0answers
29 views

What measure of test reliability is appropriate if the final test output is not the sum of item scores?

I would like to compute a measure of reliability in the sense of Cronbach's alpha, McDonalds omega or glb. However it seems to me that all these measures assume that the final test score equates to ...
2
votes
1answer
89 views

Is empirical reliability in IRT the counterpart of internal consistency in CTT?

In IRT it is not meaningful to compute internal consistency coefficients such as Cronbachs alpha, because standard error of measurement (and thus measurement precision) is not assumed uniform across ...
2
votes
2answers
91 views

Acceptable values for discrimination parameter - 2PL IRT Model

What is the acceptable range for discrimination parameters with the two-parameter IRT model? I have information from Baker (2001) but would love additional information. I have heard discrimination ...
1
vote
1answer
72 views

What does 'downward pressure on sample size' mean?

In this article, there is this phrase 'downward pressure on sample size'. This is the paragraph: no matter the approach to collecting this new data, when attempting to ...
1
vote
0answers
44 views

How can Bayesian inference improve upon logistic regression in incorporating psychometric data?

In order to estimate probability of default, banks and other financial institutions have often used logistic regression based on data involving credit history. Suppose an entrepreneur or borrower ...
0
votes
0answers
17 views

Combining different measures from different studies

I'm interested in combining outcomes from different psychiatric studies. However, the studies look at different types of diagnoses, that are measured sometimes with the same measurement, and sometimes ...
6
votes
1answer
148 views

What is Cronbach's Alpha intuitively?

I'm trying to understand Cronbach's Alpha intuitively. What is the general idea behind this construct? What properties were they trying to ensure that it had?
3
votes
0answers
44 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
1
vote
1answer
27 views

Can the test information function be used for a field test item bank?

I am field testing a few forms of items (with embedded anchor items) and will be analyzing the items using IRT, a 2PL model. These items will likely be used to create an item bank that can then create ...
1
vote
0answers
65 views

What do I do when my second order CFA model is not identified?

I am assessing a measurement model of a three-factor organisational commitment scale for my PhD project. Although congeneric single factor and first order CFA models have produced excellent model fit ...