Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

0
votes
0answers
6 views

what is the difference between generalizability theory and power analysis

Genearalizability theory, especially the decision study(D study), sounds very similar to Power analysis. A D study uses the variance components from previous variance composition of the measurement ...
1
vote
0answers
10 views

(Psychology research) Is it appropriate or coherent to standardize these variables?

I am doing a project where I'm analyzing various lexical dimensions of speech from folktales from 10 countries. And the program, LIWC, returns a value for each category with that value representing ...
0
votes
0answers
7 views

How to test if the difference between two logit-function-fits is significant or not?

I am considering a binomial experiment where each trial can only have two possible outputs (0,1). I vary the value of a predictor (experiment parameter) and repeat the experiment several times for ...
0
votes
0answers
14 views

How can I account for multiple standardized tests being used during a longitudinal study?

I am currently conducting a longitudinal study of the growth of student test scores over time, and many of the districts in my study have switched to a new state assessment. The new assessment and the ...
2
votes
0answers
33 views

Appropriateness of Rasch for formative measurement models

I've spent several months thinking about the issue of whether or not it is appropriate to apply Rasch models to formative measurement models, and I'm looking to see whether anybody else has considered ...
1
vote
0answers
11 views

intraclass correlation (ICC) for level of comprehension of individual items by different raters

I'm currently doing a translation and adaptation of a questionnaire composed of 36 items. After finishing the translation, synthesis and back-translation steps, 40 people randomly selected from a ...
0
votes
0answers
30 views

Principal Component Analysis: Scale reliability and validity

I'm using a popular psychometric scale, the Domain Specific Risk Taking (DoSpeRT) scale, that contains 30 items that measure an individual's risk taking capacity across five different dimensions: ...
0
votes
0answers
10 views

“*usefulness*” is a bivariate property used in Regression and Anova. Has a generalization (trivariate) analogon been discussed?

Just for selfstudy/exercising of algorithms I looked at the computation of the "usefulness"-measure in multiple regression, which means the part of variance which one independent item contributes to ...
3
votes
0answers
39 views

Does weighted factor analysis exist?

I would like to perform factor analysis on a set of responses to a psychological questionnaire. Is there some method that allows me to "weight" each questionnaire item by how important I believe it is ...
2
votes
1answer
33 views

Cronbach's alpha vs. Average Item-Sum correlation?

I have heard that Cronbach's alpha is used to measure reliability of a test (to check if a set of items/question are measuring the same construct). How does the Cronbach's alpha compare with the ...
0
votes
0answers
25 views

Scaled vs. unscaled Chi-square statistic in CFA / GRM fit evaluation

I generally want to assess the fit of a CFA model with ordered categorical indicators (the graded response model, GRM). There are two versions of this model, the normal-ogive GRM (ordered CFA with ...
0
votes
1answer
30 views

What is a design matrix? How do I build a design matrix for Rasch analysis?

I am trying to do an LRSM Rasch analysis of 72-item data. This data has 67 binary items and 5 ternary items. I need to create a "design matrix" in order to analyze this data. I do not really know what ...
1
vote
1answer
12 views

Randomization scheme for rating images and assessing inter-rater reliability

Apologies for the basic question, but I'm new to the area and I'm working on a research project on rating images, and assessing the reliability of these ratings, with the following structure: There ...
0
votes
0answers
46 views

convergence warning in glmer(),

Ok I have a dataset which is looking at choice behavior of rats in an olfactory decision-making task. In this task the rat has to sniff at a center odor port and depending on which odor stimulus it ...
2
votes
1answer
24 views

Correspondence between two different depression scales?

I have become involved in a project where psychiatrists are studying the correspondence between two scales measuring the level of depression among a specific patient group. One of the scales is for ...
0
votes
1answer
67 views

Average inter-item correlations as internal consistency indicator

In literature, the average inter-item correlations in the range of 0.15 to 0.50 is considered as an indicator for an acceptable level of consistency. Why do the high average inter-item correlations ...
1
vote
0answers
37 views

How can measure be unreliable but valid?

Some previous research found evidence for the predictive validity of a measure. I've done some work and found the measure to be unreliable. Since reliability provides an upper limit for predictive ...
0
votes
0answers
8 views

How to form multi-item scales after doing a factor analysis

I am doing a content analysis research for bringing out news frames, and decided use Semetko and Valkenburg (2000) method, which consists in developing a set of questions that will help measure ...
0
votes
0answers
26 views

How to equate the cut scores

I have two different tests with two different cut scores. For example a test has a cut score of 50 and the other test has a cut score 55. I want to equate the cut scores for two test. I have ...
0
votes
0answers
14 views

comparing SEM in Z and T scores

I have a function that gives me the SEM of my test for different thetas. This - I believe - is a SEM of Z scores. ...
1
vote
0answers
38 views

how to remove outliers prior to multiple imputation

A colleague came to me with the following problem. She has a complex, multivariate data set, in which respondents completed a number of measures with anywhere from 6 to 30 Likert type items for each ...
0
votes
0answers
69 views

How to calculate discrimination index and difficulty index of short answer test items?

How to calculate discrimination index and difficulty index of short answer test items? Should I use t test or the formula upper group score - lower group score divided by number of students in a group?...
1
vote
0answers
22 views

Select and weigh questions on a reading assessment

I am a phd student in computer science and as such I am the goto guy for anything "mathy" in the cross discipline research group to which I belong. I have recently been given an assignment at work I'm ...
1
vote
0answers
9 views

Inter-raters reliability for sentiment analysis lexicon

I have a list of words (more than 2000) which were rated by 4 raters for sentiments like anger (scale from 1 to 5). I run an alpha test on SPSS which gives me a coefficient of >8 which shows a good ...
2
votes
2answers
267 views

What is the two alternative forced choice paradigm

Is a two alternative forced choice paradigm (2AFC) an experimental design?
1
vote
1answer
45 views

raising the probability of correct answers of nextItem (catR)

Using the catR package I am trying to create a nextItem call that respondents would answer correctly about 65% of the time (even if that means the method is not as efficient as it could be). Children ...
0
votes
0answers
33 views

Test-retest reliability

I have come across following table in a publication, where r is correlation coefficient. I could not find detailed procedures for test-retest reliability, is it any different from paired t-test ? ...
0
votes
0answers
26 views

Calculate within and between group correlation

MY DATA: A surveys which contains 2 different questionnaires on the same subject which were taken by n=800 patients (i.e., timepoint) (#1 questionnaire: 5 questions which reach a total score of 0-42, ...
0
votes
0answers
27 views

Statistical Method for Normalization of Data

I collected data using a pre-test instrument to capture baseline emotions (two dimensions: valence and arousal) of participants. The two dimensions in the questionnaire range from calm to excited (...
3
votes
0answers
25 views

Comparing confirmatory factor analysis models with different number of observed variables

I have a 14 item scale that is hypothesized to measure self-esteem. The first model I specify uses all 14 items and predicts one latent factor (i.e., self-esteem). I would then like to specify a ...
4
votes
1answer
287 views

What is the difference between formative and reflective measurement models?

Can anyone provide examples as to why choose one over the other? Are they calculated diferently?
1
vote
0answers
20 views

CFA model with systematic temporal correlation between measures

I'm looking to fit CFA models to data in the case where measures have systematic temporal correlations between them. I'm familiar with autoregressive approaches, such as those described on Dave Kenny'...
3
votes
1answer
59 views

Non-Significant difference between individual items, but significant difference between overall scores

Say that we have a scale measuring Healthiness of Diet, and want to see if two populations have different levels of Healthiness of Diet. If we find that the difference between the two samples at the ...
9
votes
3answers
251 views

Is it always better to extract more factors when they exist?

Unlike principal components analysis, the solutions to factor analysis models are not necessarily nested. That is, the loadings (for example) for the first factor won't necessarily be identical when ...
0
votes
1answer
80 views

Detecting Differential Item Functioning (DIF) using ordered logistic regression

I am in a situation in which I want to investigate whether test items of a questionnaire appear to have DIF. The items are polytomous and ranges from 1 to 5, with 5 being the most positive answer. I'...
2
votes
0answers
27 views

What measure of test reliability is appropriate if the final test output is not the sum of item scores?

I would like to compute a measure of reliability in the sense of Cronbach's alpha, McDonalds omega or glb. However it seems to me that all these measures assume that the final test score equates to ...
2
votes
1answer
86 views

Is empirical reliability in IRT the counterpart of internal consistency in CTT?

In IRT it is not meaningful to compute internal consistency coefficients such as Cronbachs alpha, because standard error of measurement (and thus measurement precision) is not assumed uniform across ...
2
votes
2answers
81 views

Acceptable values for discrimination parameter - 2PL IRT Model

What is the acceptable range for discrimination parameters with the two-parameter IRT model? I have information from Baker (2001) but would love additional information. I have heard discrimination ...
1
vote
1answer
68 views

What does 'downward pressure on sample size' mean?

In this article, there is this phrase 'downward pressure on sample size'. This is the paragraph: no matter the approach to collecting this new data, when attempting to ...
1
vote
0answers
44 views

How can Bayesian inference improve upon logistic regression in incorporating psychometric data?

In order to estimate probability of default, banks and other financial institutions have often used logistic regression based on data involving credit history. Suppose an entrepreneur or borrower ...
0
votes
0answers
16 views

Combining different measures from different studies

I'm interested in combining outcomes from different psychiatric studies. However, the studies look at different types of diagnoses, that are measured sometimes with the same measurement, and sometimes ...
6
votes
1answer
135 views

What is Cronbach's Alpha intuitively?

I'm trying to understand Cronbach's Alpha intuitively. What is the general idea behind this construct? What properties were they trying to ensure that it had?
3
votes
0answers
39 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
1
vote
1answer
26 views

Can the test information function be used for a field test item bank?

I am field testing a few forms of items (with embedded anchor items) and will be analyzing the items using IRT, a 2PL model. These items will likely be used to create an item bank that can then create ...
1
vote
0answers
64 views

What do I do when my second order CFA model is not identified?

I am assessing a measurement model of a three-factor organisational commitment scale for my PhD project. Although congeneric single factor and first order CFA models have produced excellent model fit ...
0
votes
1answer
26 views

measure development - removing items from an item pool due to ceiling effect

During the early stages of scale development, many items within our item pool were found to have a strong ceiling effect with low score variability. In order to attempt create a meaningful measure ...
2
votes
3answers
277 views

“Two stage” factor Analysis: factoring saved factor scores

I have a model which consists of 19 questions, which are divided in three factors (factor 1 - nine questions, factor 2 - six questions, factor 3 - four questions). For this I did a factor analysis and ...
4
votes
0answers
59 views

What's an intuitive way to explain the different types of validity?

Specifically I'm thinking of a simplified division whereby validity is divided into: Construct validity 1a. Convergent validity 1b. Discriminant validity Criterion related validity 2a. Predictive ...
4
votes
2answers
110 views

The valid variance components to include in a generalizability theory model (anova) involving nesting? Are these correct interactions?

My question is about generalizability theory. A study by Bergeron, Floyd, McCormack, and Farmer (2008) is only adding to this confusion. Basically, in G-theory you want the first model you run to ...
0
votes
0answers
254 views

Interpreting summary of ICC results in R package “psych”

I have a dataframe "all.st.icc", where 27 subjects have measures of social behavior for each of 8 different years. First four rows of data are as follows: ...