Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

learn more… | top users | synonyms

4
votes
1answer
224 views

Omega H asymptotic

In the omega function in the psych package for R, we have several outputs including Hierarchical omega, Asymptotic omega and Total omega. I was reading about the outputs, but I find nothing about ...
3
votes
1answer
184 views

Validation of a questionnaire in a new population

I have 400 responses to a 20 item questionnaire which purports to measure an attitudinal constuct in medical students. The instrument was validated in the US for a single year of medical students and ...
1
vote
1answer
27 views

Where I can find resources to learn how to calculate the sample size representativeness, and realiability and validity of questionnaires?

I'm totally newbie in psychometrics but starting from a research paper with full data I would like to understand: how to calculate if the sample size is representative; how to calculate reliability ...
5
votes
0answers
98 views

How to quickly identify participants responding randomly to self-report psychometric tests with many items?

Many psychological studies involve getting participants to answer a hundred or more closed ended questions. A standard context would be a personality test with 100 items where each item is answered on ...
4
votes
0answers
43 views

What's an intuitive way to explain the different types of validity?

Specifically I'm thinking of a simplified division whereby validity is divided into: Construct validity 1a. Convergent validity 1b. Discriminant validity Criterion related validity 2a. Predictive ...
3
votes
0answers
10 views

Inter-rater agreement of a gold standard dataset - a ceiling for reliable evaluation of algorithms?

In my field, a dated gold standard dataset is used to track progress in algorithm development. Now when the state-of-the-art algorithms obtain higher correlation than is the inter-rater agreement of ...
3
votes
0answers
126 views

How high is too high with Cronbach's alpha?

Someone in my organization just sent me a scale they developed (modified items from five separate scales used in published research) to measure an employee's readiness for change. They tested the ...
3
votes
0answers
276 views

What is the relationship between scale reliability measures (Cronbach's alpha etc.) and component/factor loadings?

Let's say I have a dataset with scores on a bunch of questionnaire items, which are theoretically comprised of a smaller number of scales, like in psychology research. I know a common approach here ...
3
votes
0answers
104 views

How can IRT-Models be understood in GLM/ SEM Framework? (Predict Learning with added Paradata-Covariates)

I'll be working with data from an intelligent tutor system similar to one studied in the KDD-Cup 2010 on student performance prediction and plan to use IRT models to infer item and ability parameters. ...
3
votes
0answers
123 views

Should I run measurement invariance if confirmatory factor analysis is not affirmative?

I have a repeated measures design for a Likert scale with 10 items. This scale is assumed to be unidimensional and this is one of the research questions along with the scales measurement invariance ...
3
votes
0answers
399 views

Dealing with poor fit in an Item Response Theory model

I'm studying an online course with about 3000 students who each took several quizzes and I'm trying to apply Item Response Theory (using the ltm package in R) to model the questions, determine which ...
3
votes
0answers
2k views

How to carry out a Likert scale analysis?

Context: My research aims to assess whether parents who have been involved in a relationship with a social worker are at higher risk of child abuse/neglect. I am trying to establish whether there ...
2
votes
0answers
26 views

Combining different raters scores

In my experiment two raters evaluate 100 products, both expressing an "innovation score" on a likert scale (range is 1 to 5, where 1 is "not innovative" and 5 is "disruptive innovation"). I ...
2
votes
0answers
21 views

Estimating and using reliability metric with the same data

I'd like to know if it is reasonable to use the same set of data to estimate and use a reliability metric from repeated measures? I am mostly interesting by the minimal detectable change (MDC). But I ...
2
votes
0answers
48 views

Threshold for psychometric experiment with three (or more) possible choices

I'm running an experiment where subjects are viewing a stimulus, and they have to decide if the stimulus is brighter/darker/identical in respect to a standard image. Stimulus luminosity actually ...
2
votes
0answers
53 views

How to deal with floor effect

I am in the process of validating a five-items scale for measuring dependence on substance 'X'. I have collected data from 98 people who used the substance under consideration at least once weekly for ...
2
votes
0answers
74 views

Is it appropriate to assess internal consistency on related items measured with different response formats within the same instrument?

In my survey, I grouped 67 unique counseling techniques that I thought were related to one another (based on theory and conceptual similarity) into six "counseling domains" using fairly intuitive, ...
2
votes
0answers
134 views

Testing graded response model thresholds for significance?

I have a data set in which two raters have each rated N samples using a 5-point ordinal rating scale. My primary interest is in whether these two raters make significantly different use of those ...
2
votes
0answers
303 views

Maximizing Cronbach alpha to select the number of variables that define a given construct

I can't seem to find this anywhere on the internet or in my books. I'm putting together a construct for an underlying variable. Is it acceptable to have a construct with two variables if that ...
2
votes
0answers
501 views

Fixing error variance in a path analysis to model measurement error in scales using sem package

I want to construct a path analysis model that can account for measurement error in totally aggregated parcels, which refer to parcels where all of the items in a scale are summed or averaged. If I am ...
1
vote
0answers
28 views

What do I do when my second order CFA model is not identified?

I am assessing a measurement model of a three-factor organisational commitment scale for my PhD project. Although congeneric single factor and first order CFA models have produced excellent model fit ...
1
vote
0answers
34 views

Which CFA model is the best fit in my translation & validation study?

I have translated and adapted a questionnaire (based on a 5 factor model; 23 items) into my native language by following ITC and MAPI guidelines. Forward / backward translation, expert panel review, ...
1
vote
0answers
16 views

Correction of data using a correlation

Suppose I have measured the outcome variable A using a (psychophysical) test that determines the ability of a subject to discriminate between two stimuli with a certain difference (the variable X). It ...
1
vote
0answers
23 views

Is there any research that compares Likert scale questions to “satisfaction” scale questions?

I'm trying to put together a questionnaire for measuring employee satisfaction with an internal process. I'd like to figure out if there is reason to prefer: ...
1
vote
0answers
30 views

Confidence interval for psychometric binomial GLM model on more than one subject in R

I'm trying to estimate the confidence interval of a psychometric curve (binomial probit GLM), for a population (now only two subjects). Suppose I've subject "a" and subject "b", which performs ...
1
vote
0answers
27 views

Item Information Distorting Scale

I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...
1
vote
0answers
25 views

Factor analysis without theoretical congruence

I've been working with certain personality theory, which I'm trying to measure in a sample of students. It's a foreign theory, so I created an instrument for my specific population. I got a Cronbach's ...
1
vote
0answers
40 views

Measure difference in answers on survey depending on item response options

For my thesis I sent out a questionnaire that has 10 questions. I set defaults: the first seven question had the default set to the desired answer and the last three questions had the default set to ...
1
vote
0answers
18 views

Is there an extension of multinomial process models that can model several dependent variables?

In the setting of psychological experiments, where a categorical response has to be given, the same response can, in theory, be generated by different latent processes. For example, in an experiment ...
1
vote
0answers
478 views

Intra- and inter- observer error, and the intra-class correlation coefficient (ICC)

I have 3 observers, who each perform multiple observations in which they count a phenomenon. I'm trying to quantify both inter- and intra- observer error for this study. I'm new to this and am finding ...
1
vote
0answers
45 views

Map a normal distribution N(x,s) to an ordinal response variable

I am working on a simulation. I am going to extract a series of normally distributed values from a distribution $N(x,s)$ whose mean $x$ and variance $s$ is known. I want to pretend that such ...
1
vote
0answers
150 views

Confirmatory factor analysis for a newly constructed questionnaire

I am about to develop a new questionnaire with 142 questions I have performed exploratory factor analysis and 30 latent factors were extracted. I was wondering if it is necessary to perform ...
1
vote
0answers
96 views

how to analyze necessary/sufficient relationship?

Good day! I'm a second year psych grad student. From my fascination with formal logic, I wondered (just personal; not homework) about how to statistically establish necessary/sufficient ...
1
vote
0answers
165 views

What coefficient could I use to calculate the relative difficulty of a test in relation to others using only mean and population standard deviation?

I have a series of tests, all of them of different difficulty, and from each of them I get an average score and a population standard deviation; e.g: ...
1
vote
0answers
569 views

Which correlation analysis is appropriate for binary and ordinal variables?

Violin players were asked to rate 5 violins from least to most rich on a [0 1], 0.05-increment scale, ...
1
vote
0answers
721 views

How to validate a survey when using Likert scale?

I have developed a very simple survey to measure the attitudes of students toward science fairs. It makes use of the Likert Scale and Open Ended Questions. Do you have a recommendation for the type ...
1
vote
0answers
347 views

Obtaining factor scores from a factor analysis of mixed types (both ordinal and scale) variables

I have to analyze a data set having both ordinal and scale variables. What I was trying to do was to factor analyze the data. The data was in SPSS. But I think SPSS factor analyzes the data by ...
1
vote
0answers
302 views

How is the proof that the Quartimax/Varimax-rotation converges?

Empirically the quartimax-/varimax-rotation has proven useful and it was always converging in my applications. But from my readings years ago I have a vague remembering, that the fact of a proof of ...
0
votes
0answers
17 views

Factor analysis - Multiple field test forms with anchors

I'm hoping to conduct a confirmatory factor analysis with some field test data I have. A total of over 1450 students participated in the testing. All of the students took one live assessment and one ...
0
votes
0answers
68 views

Interpreting summary of ICC results in R package “psych”

I have a dataframe "all.st.icc", where 27 subjects have measures of social behavior for each of 8 different years. First four rows of data are as follows: ...
0
votes
0answers
43 views

Is the use of PCA appropriate to validate a designed questionnaire?

I looked at the questions that may already have my answer but unfortunately it is not the case. I would like to ask again my question which has been revised to fit the rules. I am to measure 5 ...
0
votes
0answers
16 views

The differences between criterion validity and operational validity

In some psychological articles, I have seen the term "operational validity". It is often used instead of "criterion validity". What are the differences between these two concepts?
0
votes
0answers
36 views

Attitude measurement – Score classification – Why using percentiles?

Once we have raw scores from a group of Likert scale items, why are percentiles used instead of considering all the possible scores from the total number items when classifying results of individuals ...
0
votes
0answers
25 views

Is a representative sample necessary to assess construct validity?

To generalize results to a greater population, a representative sample is necessary. In developing a psychometrically valid instrument, via confirmatory factor analysis or item response theory, we are ...
0
votes
0answers
42 views

How to build a perceptual scale using ratio measurements?

I have several physical samples of a material that vary according to some physical measure, e.g. hardness, roughness, brightness etc. Let's use brightness as the example. Now, I am not so much ...
0
votes
0answers
66 views

Identifying the number of conditions by which variance components will be divided in Generalizability Theory using QME R package

I'm using QME R Package (can be downloaded from here) to calculate Generalizability ...
0
votes
0answers
71 views

Estimates of regression coefficients at subject- vs group-level

For each of several subjects, I have mean reaction times for each of several stimuli. For each individual subject, I want to regress the RTs (as Y) against the stimuli (as X) and see if I get a slope ...
0
votes
0answers
62 views

Correlation significance test with standardized z scores

I am using spss I am looking at a criterion-validity concurrent design. I had to transform two groups performance ratings into z scores because there were differences in how raters rated them on the ...
0
votes
0answers
25 views

theoretical profile correlation

My data (n > 4000) contains the results of two separate tests (instruments). I have a profile made from likert scales for one of the instruments (S1, S2, ... S20). I want to find the corresponding ...
0
votes
0answers
533 views

Using Rasch model to explain relationships between a set of dependent and independent variables

My research study is in development economics. My data consist of more than one independent variables (continuous and categorical) as well as more than one dependent variables (categorical 5-point ...