Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

5
votes
0answers
143 views

How to quickly identify participants responding randomly to self-report psychometric tests with many items?

Many psychological studies involve getting participants to answer a hundred or more closed ended questions. A standard context would be a personality test with 100 items where each item is answered on ...
4
votes
0answers
59 views

What's an intuitive way to explain the different types of validity?

Specifically I'm thinking of a simplified division whereby validity is divided into: Construct validity 1a. Convergent validity 1b. Discriminant validity Criterion related validity 2a. Predictive ...
3
votes
0answers
35 views

Does weighted factor analysis exist?

I would like to perform factor analysis on a set of responses to a psychological questionnaire. Is there some method that allows me to "weight" each questionnaire item by how important I believe it is ...
3
votes
0answers
23 views

Comparing confirmatory factor analysis models with different number of observed variables

I have a 14 item scale that is hypothesized to measure self-esteem. The first model I specify uses all 14 items and predicts one latent factor (i.e., self-esteem). I would then like to specify a ...
3
votes
0answers
33 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
3
votes
0answers
134 views

How can IRT-Models be understood in GLM/ SEM Framework? (Predict Learning with added Paradata-Covariates)

I'll be working with data from an intelligent tutor system similar to one studied in the KDD-Cup 2010 on student performance prediction and plan to use IRT models to infer item and ability parameters. ...
3
votes
0answers
137 views

Should I run measurement invariance if confirmatory factor analysis is not affirmative?

I have a repeated measures design for a Likert scale with 10 items. This scale is assumed to be unidimensional and this is one of the research questions along with the scales measurement invariance ...
3
votes
0answers
445 views

Dealing with poor fit in an Item Response Theory model

I'm studying an online course with about 3000 students who each took several quizzes and I'm trying to apply Item Response Theory (using the ltm package in R) to model the questions, determine which ...
3
votes
0answers
2k views

How to carry out a Likert scale analysis?

Context: My research aims to assess whether parents who have been involved in a relationship with a social worker are at higher risk of child abuse/neglect. I am trying to establish whether there ...
2
votes
0answers
27 views

What measure of test reliability is appropriate if the final test output is not the sum of item scores?

I would like to compute a measure of reliability in the sense of Cronbach's alpha, McDonalds omega or glb. However it seems to me that all these measures assume that the final test score equates to ...
2
votes
0answers
57 views

Combining different raters scores

In my experiment two raters evaluate 100 products, both expressing an "innovation score" on a likert scale (range is 1 to 5, where 1 is "not innovative" and 5 is "disruptive innovation"). I ...
2
votes
0answers
24 views

Estimating and using reliability metric with the same data

I'd like to know if it is reasonable to use the same set of data to estimate and use a reliability metric from repeated measures? I am mostly interesting by the minimal detectable change (MDC). But I ...
2
votes
0answers
61 views

Threshold for psychometric experiment with three (or more) possible choices

I'm running an experiment where subjects are viewing a stimulus, and they have to decide if the stimulus is brighter/darker/identical in respect to a standard image. Stimulus luminosity actually ...
2
votes
0answers
67 views

How to deal with floor effect

I am in the process of validating a five-items scale for measuring dependence on substance 'X'. I have collected data from 98 people who used the substance under consideration at least once weekly for ...
2
votes
0answers
628 views

Intra- and inter- observer error, and the intra-class correlation coefficient (ICC)

I have 3 observers, who each perform multiple observations in which they count a phenomenon. I'm trying to quantify both inter- and intra- observer error for this study. I'm new to this and am finding ...
2
votes
0answers
79 views

Is it appropriate to assess internal consistency on related items measured with different response formats within the same instrument?

In my survey, I grouped 67 unique counseling techniques that I thought were related to one another (based on theory and conceptual similarity) into six "counseling domains" using fairly intuitive, ...
2
votes
0answers
144 views

Testing graded response model thresholds for significance?

I have a data set in which two raters have each rated N samples using a 5-point ordinal rating scale. My primary interest is in whether these two raters make significantly different use of those ...
2
votes
0answers
318 views

Maximizing Cronbach alpha to select the number of variables that define a given construct

I can't seem to find this anywhere on the internet or in my books. I'm putting together a construct for an underlying variable. Is it acceptable to have a construct with two variables if that ...
2
votes
0answers
579 views

Fixing error variance in a path analysis to model measurement error in scales using sem package

I want to construct a path analysis model that can account for measurement error in totally aggregated parcels, which refer to parcels where all of the items in a scale are summed or averaged. If I am ...
2
votes
0answers
348 views

How is the proof that the Quartimax/Varimax-rotation converges?

Empirically the quartimax-/varimax-rotation has proven useful and it was always converging in my applications. But from my readings years ago I have a vague remembering, that the fact of a proof of ...
1
vote
0answers
35 views

How can measure be unreliable but valid?

Some previous research found evidence for the predictive validity of a measure. I've done some work and found the measure to be unreliable. Since reliability provides an upper limit for predictive ...
1
vote
0answers
30 views

how to remove outliers prior to multiple imputation

A colleague came to me with the following problem. She has a complex, multivariate data set, in which respondents completed a number of measures with anywhere from 6 to 30 Likert type items for each ...
1
vote
0answers
22 views

Select and weigh questions on a reading assessment

I am a phd student in computer science and as such I am the goto guy for anything "mathy" in the cross discipline research group to which I belong. I have recently been given an assignment at work I'm ...
1
vote
0answers
7 views

Inter-raters reliability for sentiment analysis lexicon

I have a list of words (more than 2000) which were rated by 4 raters for sentiments like anger (scale from 1 to 5). I run an alpha test on SPSS which gives me a coefficient of >8 which shows a good ...
1
vote
0answers
17 views

CFA model with systematic temporal correlation between measures

I'm looking to fit CFA models to data in the case where measures have systematic temporal correlations between them. I'm familiar with autoregressive approaches, such as those described on Dave ...
1
vote
0answers
43 views

How can Bayesian inference improve upon logistic regression in incorporating psychometric data?

In order to estimate probability of default, banks and other financial institutions have often used logistic regression based on data involving credit history. Suppose an entrepreneur or borrower ...
1
vote
0answers
61 views

What do I do when my second order CFA model is not identified?

I am assessing a measurement model of a three-factor organisational commitment scale for my PhD project. Although congeneric single factor and first order CFA models have produced excellent model fit ...
1
vote
0answers
68 views

Which CFA model is the best fit in my translation & validation study?

I have translated and adapted a questionnaire (based on a 5 factor model; 23 items) into my native language by following ITC and MAPI guidelines. Forward / backward translation, expert panel review, ...
1
vote
0answers
20 views

Correction of data using a correlation

Suppose I have measured the outcome variable A using a (psychophysical) test that determines the ability of a subject to discriminate between two stimuli with a certain difference (the variable X). It ...
1
vote
0answers
30 views

Is there any research that compares Likert scale questions to “satisfaction” scale questions?

I'm trying to put together a questionnaire for measuring employee satisfaction with an internal process. I'd like to figure out if there is reason to prefer: ...
1
vote
0answers
44 views

Confidence interval for psychometric binomial GLM model on more than one subject in R

I'm trying to estimate the confidence interval of a psychometric curve (binomial probit GLM), for a population (now only two subjects). Suppose I've subject "a" and subject "b", which performs ...
1
vote
0answers
32 views

Item Information Distorting Scale

I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...
1
vote
0answers
29 views

Factor analysis without theoretical congruence

I've been working with certain personality theory, which I'm trying to measure in a sample of students. It's a foreign theory, so I created an instrument for my specific population. I got a Cronbach's ...
1
vote
0answers
67 views

Measure difference in answers on survey depending on item response options

For my thesis I sent out a questionnaire that has 10 questions. I set defaults: the first seven question had the default set to the desired answer and the last three questions had the default set to ...
1
vote
0answers
23 views

Is there an extension of multinomial process models that can model several dependent variables?

In the setting of psychological experiments, where a categorical response has to be given, the same response can, in theory, be generated by different latent processes. For example, in an experiment ...
1
vote
0answers
50 views

Map a normal distribution N(x,s) to an ordinal response variable

I am working on a simulation. I am going to extract a series of normally distributed values from a distribution $N(x,s)$ whose mean $x$ and variance $s$ is known. I want to pretend that such ...
1
vote
0answers
163 views

Confirmatory factor analysis for a newly constructed questionnaire

I am about to develop a new questionnaire with 142 questions I have performed exploratory factor analysis and 30 latent factors were extracted. I was wondering if it is necessary to perform ...
1
vote
0answers
108 views

how to analyze necessary/sufficient relationship?

Good day! I'm a second year psych grad student. From my fascination with formal logic, I wondered (just personal; not homework) about how to statistically establish necessary/sufficient ...
1
vote
0answers
218 views

What coefficient could I use to calculate the relative difficulty of a test in relation to others using only mean and population standard deviation?

I have a series of tests, all of them of different difficulty, and from each of them I get an average score and a population standard deviation; e.g: ...
1
vote
0answers
671 views

Which correlation analysis is appropriate for binary and ordinal variables?

Violin players were asked to rate 5 violins from least to most rich on a [0 1], 0.05-increment scale, ...
1
vote
0answers
873 views

How to validate a survey when using Likert scale?

I have developed a very simple survey to measure the attitudes of students toward science fairs. It makes use of the Likert Scale and Open Ended Questions. Do you have a recommendation for the type ...
1
vote
0answers
394 views

Obtaining factor scores from a factor analysis of mixed types (both ordinal and scale) variables

I have to analyze a data set having both ordinal and scale variables. What I was trying to do was to factor analyze the data. The data was in SPSS. But I think SPSS factor analyzes the data by ...
0
votes
0answers
21 views

Principal Component Analysis: Scale reliability and validity

I'm using a popular psychometric scale, the Domain Specific Risk Taking (DoSpeRT) scale, that contains 30 items that measure an individual's risk taking capacity across five different dimensions: ...
0
votes
0answers
9 views

“*usefulness*” is a bivariate property used in Regression and Anova. Has a generalization (trivariate) analogon been discussed?

Just for selfstudy/exercising of algorithms I looked at the computation of the "usefulness"-measure in multiple regression, which means the part of variance which one independent item contributes to ...
0
votes
0answers
21 views

Scaled vs. unscaled Chi-square statistic in CFA / GRM fit evaluation

I generally want to assess the fit of a CFA model with ordered categorical indicators (the graded response model, GRM). There are two versions of this model, the normal-ogive GRM (ordered CFA with ...
0
votes
0answers
35 views

convergence warning in glmer(),

Ok I have a dataset which is looking at choice behavior of rats in an olfactory decision-making task. In this task the rat has to sniff at a center odor port and depending on which odor stimulus it ...
0
votes
0answers
8 views

How to form multi-item scales after doing a factor analysis

I am doing a content analysis research for bringing out news frames, and decided use Semetko and Valkenburg (2000) method, which consists in developing a set of questions that will help measure ...
0
votes
0answers
24 views

How to equate the cut scores

I have two different tests with two different cut scores. For example a test has a cut score of 50 and the other test has a cut score 55. I want to equate the cut scores for two test. I have ...
0
votes
0answers
14 views

comparing SEM in Z and T scores

I have a function that gives me the SEM of my test for different thetas. This - I believe - is a SEM of Z scores. ...
0
votes
0answers
48 views

How to calculate discrimination index and difficulty index of short answer test items?

How to calculate discrimination index and difficulty index of short answer test items? Should I use t test or the formula upper group score - lower group score divided by number of students in a ...