Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

0
votes
0answers
21 views

Principal Component Analysis: Scale reliability and validity

I'm using a popular psychometric scale, the Domain Specific Risk Taking (DoSpeRT) scale, that contains 30 items that measure an individual's risk taking capacity across five different dimensions: ...
0
votes
0answers
9 views

“*usefulness*” is a bivariate property used in Regression and Anova. Has a generalization (trivariate) analogon been discussed?

Just for selfstudy/exercising of algorithms I looked at the computation of the "usefulness"-measure in multiple regression, which means the part of variance which one independent item contributes to ...
3
votes
0answers
35 views

Does weighted factor analysis exist?

I would like to perform factor analysis on a set of responses to a psychological questionnaire. Is there some method that allows me to "weight" each questionnaire item by how important I believe it is ...
2
votes
1answer
29 views

Cronbach's alpha vs. Average Item-Sum correlation?

I have heard that Cronbach's alpha is used to measure reliability of a test (to check if a set of items/question are measuring the same construct). How does the Cronbach's alpha compare with the ...
0
votes
0answers
21 views

Scaled vs. unscaled Chi-square statistic in CFA / GRM fit evaluation

I generally want to assess the fit of a CFA model with ordered categorical indicators (the graded response model, GRM). There are two versions of this model, the normal-ogive GRM (ordered CFA with ...
0
votes
1answer
28 views

What is a design matrix? How do I build a design matrix for Rasch analysis?

I am trying to do an LRSM Rasch analysis of 72-item data. This data has 67 binary items and 5 ternary items. I need to create a "design matrix" in order to analyze this data. I do not really know what ...
1
vote
1answer
12 views

Randomization scheme for rating images and assessing inter-rater reliability

Apologies for the basic question, but I'm new to the area and I'm working on a research project on rating images, and assessing the reliability of these ratings, with the following structure: There ...
0
votes
0answers
35 views

convergence warning in glmer(),

Ok I have a dataset which is looking at choice behavior of rats in an olfactory decision-making task. In this task the rat has to sniff at a center odor port and depending on which odor stimulus it ...
2
votes
1answer
24 views

Correspondence between two different depression scales?

I have become involved in a project where psychiatrists are studying the correspondence between two scales measuring the level of depression among a specific patient group. One of the scales is for ...
0
votes
1answer
50 views

Average inter-item correlations as internal consistency indicator

In literature, the average inter-item correlations in the range of 0.15 to 0.50 is considered as an indicator for an acceptable level of consistency. Why do the high average inter-item correlations ...
1
vote
0answers
35 views

How can measure be unreliable but valid?

Some previous research found evidence for the predictive validity of a measure. I've done some work and found the measure to be unreliable. Since reliability provides an upper limit for predictive ...
0
votes
0answers
8 views

How to form multi-item scales after doing a factor analysis

I am doing a content analysis research for bringing out news frames, and decided use Semetko and Valkenburg (2000) method, which consists in developing a set of questions that will help measure ...
0
votes
0answers
24 views

How to equate the cut scores

I have two different tests with two different cut scores. For example a test has a cut score of 50 and the other test has a cut score 55. I want to equate the cut scores for two test. I have ...
0
votes
0answers
14 views

comparing SEM in Z and T scores

I have a function that gives me the SEM of my test for different thetas. This - I believe - is a SEM of Z scores. ...
1
vote
0answers
30 views

how to remove outliers prior to multiple imputation

A colleague came to me with the following problem. She has a complex, multivariate data set, in which respondents completed a number of measures with anywhere from 6 to 30 Likert type items for each ...
0
votes
0answers
48 views

How to calculate discrimination index and difficulty index of short answer test items?

How to calculate discrimination index and difficulty index of short answer test items? Should I use t test or the formula upper group score - lower group score divided by number of students in a ...
1
vote
0answers
22 views

Select and weigh questions on a reading assessment

I am a phd student in computer science and as such I am the goto guy for anything "mathy" in the cross discipline research group to which I belong. I have recently been given an assignment at work I'm ...
1
vote
0answers
7 views

Inter-raters reliability for sentiment analysis lexicon

I have a list of words (more than 2000) which were rated by 4 raters for sentiments like anger (scale from 1 to 5). I run an alpha test on SPSS which gives me a coefficient of >8 which shows a good ...
2
votes
2answers
173 views

What is the two alternative forced choice paradigm

Is a two alternative forced choice paradigm (2AFC) an experimental design?
1
vote
1answer
43 views

raising the probability of correct answers of nextItem (catR)

Using the catR package I am trying to create a nextItem call that respondents would answer correctly about 65% of the time (even if that means the method is not as efficient as it could be). Children ...
0
votes
0answers
30 views

Test-retest reliability

I have come across following table in a publication, where r is correlation coefficient. I could not find detailed procedures for test-retest reliability, is it any different from paired t-test ? ...
0
votes
0answers
24 views

Calculate within and between group correlation

MY DATA: A surveys which contains 2 different questionnaires on the same subject which were taken by n=800 patients (i.e., timepoint) (#1 questionnaire: 5 questions which reach a total score of 0-42, ...
0
votes
0answers
24 views

Statistical Method for Normalization of Data

I collected data using a pre-test instrument to capture baseline emotions (two dimensions: valence and arousal) of participants. The two dimensions in the questionnaire range from calm to excited ...
3
votes
0answers
23 views

Comparing confirmatory factor analysis models with different number of observed variables

I have a 14 item scale that is hypothesized to measure self-esteem. The first model I specify uses all 14 items and predicts one latent factor (i.e., self-esteem). I would then like to specify a ...
4
votes
1answer
171 views

What is the difference between formative and reflective measurement models?

Can anyone provide examples as to why choose one over the other? Are they calculated diferently?
1
vote
0answers
17 views

CFA model with systematic temporal correlation between measures

I'm looking to fit CFA models to data in the case where measures have systematic temporal correlations between them. I'm familiar with autoregressive approaches, such as those described on Dave ...
3
votes
1answer
55 views

Non-Significant difference between individual items, but significant difference between overall scores

Say that we have a scale measuring Healthiness of Diet, and want to see if two populations have different levels of Healthiness of Diet. If we find that the difference between the two samples at the ...
9
votes
3answers
244 views

Is it always better to extract more factors when they exist?

Unlike principal components analysis, the solutions to factor analysis models are not necessarily nested. That is, the loadings (for example) for the first factor won't necessarily be identical when ...
0
votes
1answer
78 views

Detecting Differential Item Functioning (DIF) using ordered logistic regression

I am in a situation in which I want to investigate whether test items of a questionnaire appear to have DIF. The items are polytomous and ranges from 1 to 5, with 5 being the most positive answer. ...
2
votes
0answers
27 views

What measure of test reliability is appropriate if the final test output is not the sum of item scores?

I would like to compute a measure of reliability in the sense of Cronbach's alpha, McDonalds omega or glb. However it seems to me that all these measures assume that the final test score equates to ...
2
votes
1answer
78 views

Is empirical reliability in IRT the counterpart of internal consistency in CTT?

In IRT it is not meaningful to compute internal consistency coefficients such as Cronbachs alpha, because standard error of measurement (and thus measurement precision) is not assumed uniform across ...
0
votes
0answers
59 views

Acceptable values for discrimination parameter - 2PL IRT Model

What is the acceptable range for discrimination parameters with the two-parameter IRT model? I have information from Baker (2001) but would love additional information. I have heard discrimination ...
1
vote
1answer
66 views

What does 'downward pressure on sample size' mean?

In this article, there is this phrase 'downward pressure on sample size'. This is the paragraph: no matter the approach to collecting this new data, when attempting to ...
1
vote
0answers
44 views

How can Bayesian inference improve upon logistic regression in incorporating psychometric data?

In order to estimate probability of default, banks and other financial institutions have often used logistic regression based on data involving credit history. Suppose an entrepreneur or borrower ...
0
votes
0answers
15 views

Combining different measures from different studies

I'm interested in combining outcomes from different psychiatric studies. However, the studies look at different types of diagnoses, that are measured sometimes with the same measurement, and sometimes ...
6
votes
1answer
125 views

What is Cronbach's Alpha intuitively?

I'm trying to understand Cronbach's Alpha intuitively. What is the general idea behind this construct? What properties were they trying to ensure that it had?
3
votes
0answers
33 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
1
vote
1answer
24 views

Can the test information function be used for a field test item bank?

I am field testing a few forms of items (with embedded anchor items) and will be analyzing the items using IRT, a 2PL model. These items will likely be used to create an item bank that can then create ...
1
vote
0answers
61 views

What do I do when my second order CFA model is not identified?

I am assessing a measurement model of a three-factor organisational commitment scale for my PhD project. Although congeneric single factor and first order CFA models have produced excellent model fit ...
0
votes
1answer
25 views

measure development - removing items from an item pool due to ceiling effect

During the early stages of scale development, many items within our item pool were found to have a strong ceiling effect with low score variability. In order to attempt create a meaningful measure ...
2
votes
3answers
260 views

“Two stage” factor Analysis: factoring saved factor scores

I have a model which consists of 19 questions, which are divided in three factors (factor 1 - nine questions, factor 2 - six questions, factor 3 - four questions). For this I did a factor analysis and ...
4
votes
0answers
59 views

What's an intuitive way to explain the different types of validity?

Specifically I'm thinking of a simplified division whereby validity is divided into: Construct validity 1a. Convergent validity 1b. Discriminant validity Criterion related validity 2a. Predictive ...
4
votes
2answers
104 views

The valid variance components to include in a generalizability theory model (anova) involving nesting? Are these correct interactions?

My question is about generalizability theory. A study by Bergeron, Floyd, McCormack, and Farmer (2008) is only adding to this confusion. Basically, in G-theory you want the first model you run to ...
0
votes
0answers
232 views

Interpreting summary of ICC results in R package “psych”

I have a dataframe "all.st.icc", where 27 subjects have measures of social behavior for each of 8 different years. First four rows of data are as follows: ...
0
votes
0answers
55 views

Is the use of PCA appropriate to validate a designed questionnaire?

I looked at the questions that may already have my answer but unfortunately it is not the case. I would like to ask again my question which has been revised to fit the rules. I am to measure 5 ...
2
votes
0answers
57 views

Combining different raters scores

In my experiment two raters evaluate 100 products, both expressing an "innovation score" on a likert scale (range is 1 to 5, where 1 is "not innovative" and 5 is "disruptive innovation"). I ...
1
vote
0answers
68 views

Which CFA model is the best fit in my translation & validation study?

I have translated and adapted a questionnaire (based on a 5 factor model; 23 items) into my native language by following ITC and MAPI guidelines. Forward / backward translation, expert panel review, ...
1
vote
1answer
81 views

Questions regarding interpretation of item parameters: inconsistent results between classical item analysis and item response analysis

I was not trained as a psychometrician, so my questions may be concept-related. If you know different resources, books, or articles that can clear my confusions, please let me know. Thank you very ...
1
vote
1answer
41 views

Where I can find resources to learn how to calculate the sample size representativeness, and realiability and validity of questionnaires?

I'm totally newbie in psychometrics but starting from a research paper with full data I would like to understand: how to calculate if the sample size is representative; how to calculate reliability ...
3
votes
1answer
128 views

Is it possible to do IRT distractor analysis using R?

I've been using the mirt package to analyse a few response matrixes I have through IRT. However, every time I need to perform distractor analysis, I end up having ...