# Tagged Questions

Psychometrics has evolved as a subfield of psychology to become the science of measurement of unobservable individual characteristics.

3answers
10k views

### Factor analysis of questionnaires composed of Likert items

I used to analyse items from a psychometric point of view. But now I am trying to analyse other types of questions on motivation and other topics. These questions are all on Likert scales. My initial ...
2answers
17k views

### Inter-rater reliability for ordinal or interval data

Which inter-rater reliability methods are most appropriate for ordinal or interval data? I believe that "Joint probability of agreement" or "Kappa" are designed for nominal data. Whilst "Pearson" and ...
2answers
14k views

### How to transform ordinal data from questionnaire into proper interval data?

Are there any straightforward methods of transforming ordinal level data into interval level (just as there are for doing it the other way round)? And performable in Excel or SPSS? Having the data, ...
4answers
15k views

### How to compute correlation between/within groups of variables?

I have a matrix of 1000 observations and 50 variables each measured on a 5-point scale. These variables are organized into groups, but there aren't an equal number of variables in each group. I'd ...
2answers
4k views

### Assessing reliability of a questionnaire: dimensionality, problematic items, and whether to use alpha, lambda6 or some other index?

I am analyzing scores given by participants attending an experiment. I want to estimate the reliability of my questionnaire which is composed of 6 items aimed at estimating the attitude of the ...
1answer
5k views

### Looking for a step through an example of a factor analysis on dichotomous data (binary variables) using R

I have some dichotomous data, only binary variables, and my boss asked me to perform a factor analysis using the tetrachoric correlations matrix. I’ve previously been able to teach myself how to run ...
4answers
10k views

### How to get started with applying item response theory and what software to use?

Context I have been reading about item response theory, and I find it fascinating. I believe I understand the basics, but I am left wondering how to apply statistical techniques related to the area. ...
3answers
6k views

### Validating questionnaires

I am designing a questionnaire for my dissertation. I am in the process of validating the questionnaire I have applied a Cronbach's alpha test to the initial sample group. The responses to the ...
5answers
2k views

### Is it appropriate to treat n-point Likert scale data as n trials from a binomial process?

I've never liked how people typically analyze data from Likert scales as if error were continuous & Gaussian when there are reasonable expectations that these assumptions are violated at least at ...
2answers
813 views

### Reliability in Elicitation Exercise

I am trying to calculate the reliability in an elicitation exercise by analysing some test-retest questions given to the experts. The experts elicited a series of probability distributions which were ...
1answer
1k views

### VSS criterion for the number of factors (in R's psych package)

I'm working from caracal's great example conducting a factor analysis on dichotomous data using R and I'm now struggling to understand the difference between VSS complexity 1 and VSS complexity 2 in ...
4answers
5k views

### Is 0 a valid value in a Likert scale?

I have carried out my pilot study on language learning motivation using a 6 point Likert scale but from 0 (strongly disagree) to 5 (very much agree). I noticed a colleague in his survey used 1 to 6. ...
1answer
150 views

### Testing the importance of an item among a finite set of items

Consider the following survey question: Q: How would you classify the importance for you of the following 5 items: A B C D E Assign to each item a number ...
6answers
2k views

### Introduction to structural equation modeling

I am asked by colleagues some help in this subject, that I don’t really know. They made hypotheses on the role of some latent variables in one study, and a referee asked them to formalize this in SEM. ...
4answers
964 views

### FA: Choosing Rotation matrix, based on “Simple Structure Criteria”

One of the most important issues in using factor analysis is its interpretation. Factor analysis often uses factor rotation to enhance its interpretation. After a satisfactory rotation, the rotated ...
1answer
2k views

### Which package to use for convergent and discriminant validity in R?

I am new to R and want to assess validity of some scales using R. Is there a package that I can use to test for convergent and discriminant validity in R?
1answer
2k views

### Interpreting AIC and BIC fit

I'm writing a CFA paper, and I have run into some trouble interpreting the AIC and BIC. This is my first paper using continuous variables, thus the first time I will be reporting these fit statistics ...
2answers
1k views

### When do you consider a variable is a latent variable?

The problem is to define when a variable might be considered as a latent variable. I am interested in how to describe a latent variable, and what are the properties of latent variables. My twofold ...
2answers
4k views

### Recommended procedure for factor analysis on dichotomous data with R

I have to run a factor analysis on a dataset made up of dichotomous variables (0=yes, 1= no) and I don´t know if I'm on the right track. Using tetrachoric() I ...
3answers
3k views

### Classification after factor analysis

I have analysed several dimensions in a survey. Each part of the survey represents a theoretical dimension and is analysed with factorial analysis. I want to use scores from factor analysis to do a ...
4answers
16k views

### What are the primary differences between z-scores and t-scores, and are they both considered standard scores?

We are currently converting student test scores in this manner : ( ScaledScore - ScaledScore Mean ) / StdDeviation ) * 15 + 100 I was referring to this ...
2answers
186 views

### What is a good method to identify outliers in exam data?

I give my students an exam that has 8 questions on it. Each question is about a particular topic. The exam is made up on the fly by randomly selecting 1 question for each topic from a pool of ...
3answers
756 views

### $\chi^2$ tests to compare the fit of large samples logistic models

Does anyone know of any $\chi^2$ tests to compare the fit of logistic models which factor out the sample size? I'm dealing with a very large sample and I fear the significant $\chi^2$ test I get when ...
1answer
126 views

### Is it possible to do IRT distractor analysis using R?

I've been using the mirt package to analyse a few response matrixes I have through IRT. However, every time I need to perform distractor analysis, I end up having ...
1answer
268 views

### Item information in IRT

According to item information curves, item information for a 2PL IRT model is $I(\theta)=a^2_i p_i(\theta) q_i(\theta)$ To determine $p_i(\theta)$ and $q_i(\theta)$, do you just use the observed ...
0answers
42 views

### How can Bayesian inference improve upon logistic regression in incorporating psychometric data?

In order to estimate probability of default, banks and other financial institutions have often used logistic regression based on data involving credit history. Suppose an entrepreneur or borrower ...
0answers
55 views

### Is the use of PCA appropriate to validate a designed questionnaire?

I looked at the questions that may already have my answer but unfortunately it is not the case. I would like to ask again my question which has been revised to fit the rules. I am to measure 5 ...
0answers
620 views

### Using Rasch model to explain relationships between a set of dependent and independent variables

My research study is in development economics. My data consist of more than one independent variables (continuous and categorical) as well as more than one dependent variables (categorical 5-point ...
3answers
2k views

### How to correctly interpret a parallel analysis in exploratory factor analysis?

Some scientific papers report results of parallel analysis of principal axis factor analysis in a way inconsistent with my understanding of the methodology. What am I missing? Am I wrong or are they. ...
3answers
6k views

### Differences between tetrachoric and Pearson correlation

Currently I'm analysing around 300 items in the field of education. I'm interested in the dimensionality of the dataset. So I compute a matrix of tetrachoric correlation. The goal is to do a factor ...
2answers
5k views

### Omega vs. alpha reliability

I wonder if someone can explain what is the main difference between omega and alpha reliabilities? I understand an omega reliability is based on hierarchical factor model as shown in the following ...
2answers
2k views

### Item Analysis for an R newbie

I am trying to assess a 20-item multliple choice test. I want to perform an item analysis such as can be found in this example. So for each question I want the P-value and the correlation with the ...
2answers
1k views

### Advice on scientifically sound scale construction

I've been given a set of 20 Likert-items (ranging from 1-5, sample size n = 299) within the field of organizational research. The items are intended to measure a latent concept which is ...
1answer
2k views

### Data transformation for Principal Components Analysis from different Likert scales

I have data from a survey comprised of several measures that used different Likert-type scaling (4-, 5-, and 6-point scales). I would like to run a principal components analysis using the data from ...
2answers
747 views

### Combining ratings from multiple raters of different accuracy

I have a data set of J raters who each give ratings to I objects (on say a scale of 1 to 5). My goal would be to construct some kind of overall rating for each i based on all of the raters' scores. ...
4answers
2k views

### Can one validly reduce the numbers of items in a published Likert-scale?

[edits made in response to feedback- thanks :-) ] Doh! More edits! Sorry! Hello- I am doing some rather rough and ready data collection with a survey sent out to healthcare staff using a published ...
3answers
13k views

### Is it acceptable to use Cronbach's alpha to assess reliability of questionnaire composed of categorical and conditional items?

Background: I have data on a questionnaire made up of categorical factual items. Some of them are binary and others have more than two categories. For example: ...
1answer
252 views

### IRT/Rasch modeling with very large N

I want to fit a 1-parameter IRT model on a questionaire with 15 questions and about six million people. Considering the large N, standard errors aren't essential. It looks like the IRT world is sort ...
3answers
3k views

### A multitrait-multimethod matrix and data set

I am working my way through Multitrait-Multimethod Matrix in a psychometrics class. We're only required to be able to analyze them but I'd really like to be able to construct them. I think I am able ...
0answers
2k views

### How to carry out a Likert scale analysis?

Context: My research aims to assess whether parents who have been involved in a relationship with a social worker are at higher risk of child abuse/neglect. I am trying to establish whether there ...
2answers
5k views

### Likert scale and yes/no calculation of one score

My survey is based on observations and interviews. The observation questions are yes/no type questions and interview questions are 4 points. $1=$ never, $2=$ sometimes, $3=$ usually, and $4=$ ...
2answers
472 views

### How is Item Response Theory (IRT) called for continuous response?

I would like to model my problem using something similar to Item Response Theory - but my responses are not binary - they are continuous in [0;1]. How are these models/the research field called?
1answer
319 views

### What am I measuring when I apply a graded response model to the “Hunting of the Snark” dataset?

In another question, I asked about the statistical validity of StackExchange's "Hunting of the Snark" dataset, and whether or not we could draw any conclusions from its results. I measured a few ...
1answer
5k views

### Cronbach's alpha negative result

What should I do if I am getting negative results for Cronbach's alpha? Is there anyway to adjust it?
1answer
9k views

### 4 point and 5 point Likert scales in a new questionnaire

I'm looking to confirm whether having a 4 point Likert Scale for some constructs, and a 5 point Likert scale for other constructs will be suitable when developing a questionnaire? I have 7 constructs ...
1answer
1k views

### Internal consistency reliability in item response theory models

How can internal consistency reliability of a test and of individual test items be quantified in Item Response Theory models? I know I can resort to Classical Test Theory, Cronbach's alpha, and other ...
1answer
490 views

### Comparing two logit or probit curves using a single parameter

I've conducted a psychological experiment on the same subject, under two different condition. For each condition I've collected the number of correct and wrong answer for each stimulus (number of ...
3answers
225 views

### Aggregate opinions to a single measure

I have a following situation in which 6 experts are asked to give opinion about a bottle of wine by rating it from 1 - 5 stars. In addition, somehow we have a measure of the informativeness of each ...
1answer
766 views

### Standard formula for quick calculation of scores

I have three datasets as follows: Dataset A can have a maximum score of 30. (Individual scores can be from 1 to 30) Dataset B can have a maximum score of 45. (Individual scores can be from 1 to 45) ...
0answers
32 views

### Item Information Distorting Scale

I'm refining a psychometric questionnaire and working to estimate (2PL) IRT parameters for the items in order to score the items. It's a small scale consisting of 5 questions. The MAP and VSS-1 ...