Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [reliability]

A measure is said to have a high reliability if it produces similar results under consistent conditions. DO NOT confuse reliability with validity (see tag wiki). DO NOT use for inter-rater reliability which has its own tag inter-rater

-1
votes
0answers
18 views

Reliability Testing using cronbach alpha [closed]

Good day am carrying out a research on 200 sample size and 20 items of question. how can i determine the coefficient level using cronbach alpha
0
votes
0answers
14 views

How i can compute Mezzich's Kappa?

Specifically, i need to compute the proportion of chance agreement in the Mezzich's Kappa formula. Thanks in advance.
0
votes
0answers
17 views

Joint reliability model

I have a following problem. There is a device where I expect exponential reliability model with parameter $\lambda$. So the probability of failure is $$P[T\ge t] = e^{-\lambda t},~t\ge 0$$. If the ...
0
votes
0answers
16 views

Test to check reliability of sample splitting procedure

As someone with little background in statistics, I am trying to test the reliability of the splitting procedure for screen analysis currently done in the lab (if the particle size distribution will be ...
1
vote
0answers
10 views

Is it correct to a total Cronbach´s alpha analysis to a tridimensional scale?

There´s a statement that says that Cronbach´s alpha assumes unidimensionality. In that sense, is it correct to calculate a global alpha? I mean, by including all 3 dimensions, and not by separating ...
1
vote
0answers
48 views

Can we statistically predict the next Lion Air incident? [closed]

Plugging the Lion Air incidents on Aviation Safety into a Crow/AMSAA tool gives MTBF of about 15 months at the moment. Is this reasonable for their fleet size of 115? Link to the: incident data ...
0
votes
0answers
16 views

How to do test-retest reliability with 3 time points?

I have tested the first two, and got a good score, but I am unsure how to do it with multiple time points. Would I conduct a test with time 1 and 2, then time 2 and 3? Or what is the best approach? ...
1
vote
0answers
11 views

Reliability of Composite Variable Made of 4 Measures?

I am conducting some analyses with a composite variable constructed by averaging 4 separate measures. I would like to know the reliability of the composite variable. Is Cronbach's alpha appropriate in ...
0
votes
0answers
20 views

How can we predict reliability for type 1 censored data?

I am working on a somewhat challenging problem. I have data for a large set of equipment. My main research question is: How many units will fail in each future month in the coming decade? So far I ...
0
votes
0answers
10 views

variable's composite reliability and Cronbach alpha is equal to and above 0.7 in model but AVE is less than 0.3 less than 0.5

If one variable's composite reliability and Cronbach alpha is equal to and above 0.7 in model but AVE is less than 0.3 less than 0.5 then at what conditions it would be ok? I found following ...
0
votes
1answer
83 views

Hazard function for a growing population

I want to model the expected failure rate in a growing population. Each individual has a high infant mortality (e.g. Weibull(shape=0.5, scale=100)). So the ...
1
vote
2answers
42 views

Book reference request for statistical reliability theory

I'm about to learn statistical reliability theory for the first time. I'm told that Barlow and Proschan's book entitled "Statistical theory of reliability and life testing" is a classic on this topic. ...
1
vote
0answers
33 views

Best easy and straightforward way of computing McDonald's omega in a unidimensional test with 'psych' in R

I'm preparing a masterclass in Psychometrics for a course, and I would like to introduce McDonald's omega coefficient as the proper index for assessing the reliability of a scale. For what I can see, ...
1
vote
0answers
146 views

McDonald's Omega: Assumptions, Coefficients and Interpretation

After browsing cross validated and several other sources on the web, I still cannot get a grip on McDonald's Omega as a measure of internal consistency. I have a hunch that many fellow social ...
1
vote
0answers
26 views

Index of within-participant consistency in two-alternative forced-choice (2AFC) task?

My study consists of 2AFC trials: Participants choose which one of a pair of stimuli has a certain quality. There are 9 pairs and 6 stimuli (2 categories of 3 stimuli each) in total. The stimuli ...
0
votes
1answer
26 views

How to calculate Mean Sample-Based Reliability Estimates Used for Analyses

I am conducting a meta analysis of Pearson's r , of correlations presented between test scores and job performance ratings. I need to present an uncorrected effect size and an effect size corrected ...
0
votes
1answer
42 views

Cronbach alpha with knowledge items?

I have a very specific question about the application of Cronbach's alpha to data with trichotomous responses (true-false-don't know). I have an acquaintance I'm helping replicate a study using a 45 ...
0
votes
0answers
20 views

Reliability in health and disease

I have a dataset consisting of 2 groups, healthy subjects and patients. Both groups did the same test twice. The groups are not even (n=20 and n= 39 respectively). I want to get the reliability of my ...
1
vote
0answers
14 views

Intraclass Correlations for Intraobserver-Reliability in Hierarchical Data

We have a dataset where multiple measurements of the same variable are taken from individuals and we want to measure the intraobserver reliability. However, the individuals each come from different ...
0
votes
0answers
69 views

Test-Retest reliability for mixed models

We have an experimental setup as follows: A set of subjects each repeat a single task multiple times in a single session, yielding a set of measurements. The same subjects then repeat the same task ...
0
votes
0answers
15 views

Cronbach's Alpha to Rate Videos

I would like to see if we can have reliable metrics that can be used to rate videos (V1:V15). I have 20 people for each video (id1:id20). And 2 metrics: ...
0
votes
0answers
24 views

Cronbach's alpha and yes-no-don't know scales

I am helping a friend analyze some questionnaire data, and I am currently a bit confused about how I should go about doing reliability analysis on her items. She is using a 45 item scale, assessing ...
0
votes
0answers
16 views

Reliability calculation using R

I have this handout about reliability calculation, but I just can't figure out how I could do this using R. Here is the simulated data. It is a questionaire that has ten quaestions and was answered ...
0
votes
0answers
13 views

Reliability and Validity of all items in a questionaire

I am stuck in finding a suitable way to report the reliability of each item within my instrument (questionnaire). The questionnaire has a couple of sections: yes/no questions, multiple choice ...
0
votes
0answers
13 views

Seeking a measure of multiple rater consistency for cateorical ratings with sparse data

I have multiple raters and sparse data. The raters can decide how many attributes apply to a person. I have many empty cells due to many attributes and few attributes actually chosen. What tools ...
5
votes
2answers
98 views

How to index rater agreement: multiple raters identify strenghts/weaknesses from 30 traits?

Six raters independently rated 25 people by identifying strengths and weaknesses: choosing among 35 traits (15 positive traits and 20 negative traits). The rater instructions said to identify a ...
1
vote
0answers
60 views

What is the point of reporting split-half reliability together with Cronbach's alpha?

In several journal articles I read, autors report split-half reliability (Guttman formula obtained if questionnaire is splited into two parts: odd items and even items, usually) together with Cronbach'...
0
votes
0answers
20 views

Derivation of equations for confidence interval of mean of a Poisson distributed variable

I'm having trouble understanding why the first equation found in the following Wikipedia link is valid: https://en.m.wikipedia.org/wiki/Poisson_distribution#Confidence_interval Can anyone here derive/...
1
vote
1answer
169 views

Using Cohen's kappa for multiple item types (e.g. binary and non-binary)

I am looking to calculate Cohen's kappa for multiple items in a scale (15 items, with two raters). The problem is that for 12 of the items the responses available to the raters were binary (yes, no). ...
0
votes
0answers
38 views

Can we use cronbach's alpha reliability test to check a variable scale with 3 question, same 7 point scale but different values?

I am a research student and doing my first paper. I am confused about checking the reliability of my adopted scale. I am using TAM as a theoretical model for my research and for it I adopted the ...
0
votes
0answers
37 views

Device that improves with time

I am unable to find an example of a device that improves with time, i.e., every time you use the device, the probability of it failing is less than before. I understand that it is an example of ...
0
votes
0answers
16 views

Test-retest Reliability Before and After

Is there some way I can assess test-retest reliability for a questionnaire based on pre and post-treatment results for patients? They are both using the same questionnaire, but the patients are in the ...
0
votes
0answers
24 views

What qualifies as a good split-half relibability?

We were doing an sentiment annotation task for natural language processing and achieved a split-half reliablity of about r=.9. I am rather confident that this is a good score, but I would like to have ...
1
vote
0answers
35 views

Analyzing satisfaction levels of PhD scholars

I am analyzing satisfaction levels of PhD scholars in a particular region. We got 7 independent variables from literature review and constructed 45 questionnaires accordingly: for example, 5 ...
1
vote
0answers
21 views

What would be an appropriate intra-rater reliability method for just 3 items test-retest

I had 20 raters evaluating 35 items with continuous interval scores. 3 items (randomly chosen out of the 35 for each rater) were tested at the beginning and then again at the end of the session. I ...
1
vote
0answers
56 views

how to calculate inter-rater agreement of nominal data with multiple categories in r

coder1 results coder2 results Hi, what method should I use to calculate inter-coder reliability between these two data in r? The row stands for the cases, and the a1~ a23/A1~A23 stands for the ...
0
votes
0answers
21 views

Weibull hazard Rate for Machine Vibration data

I have vibration measurements of machines over different duration till the Machine breaks down. I have several such records. How do I fit a Weibull model and compute the hazard rate. Is it that for ...
0
votes
0answers
11 views

Similar questions across similar samples at different times - reliability?

I'm working with pilot data where there exist several versions of the same questionnaires that were administered to different participants across various time points. I'd like to get a better sense of ...
0
votes
0answers
5 views

Scale reliability across wave

I have a 23 item scale (ordinal) that was collected at Wave 1 and Wave 2 for the same respondents. I calculated the sum of the scale after recoding the items that were worded negatively. I calculated ...
0
votes
0answers
74 views

reliability ratio in multiple linear regression with measurement error

I would like to know about reliability ratio in OLS with measurement error (simple regression) Model : ...
0
votes
0answers
19 views

Spearman's rank in R; correlation between domains with multiple variables and three overall satisfaction items

I am testing a questionnaire's reliability and congruent validity in R. My data consists of 140 observations and 34 variables (questions on a likert scale). The variables are divided into five ...
0
votes
1answer
69 views

error variance (delta) of a factor - lavaan

I'm building a measurement model with three factors and I don't know how to set the error variance ((1 - reliability) * item's variance) of one of the factors in lavaan. Thanks for your help.
2
votes
1answer
29 views

Does changing the point range of items of a construct measured through likert scale affect in any way validity and reliability of the construct?

I am using a pre-existing universal tested scale to measure a specific construct. The construct is measured through 5 items on a 5 points likert scale. However, I would probably need to change the ...
4
votes
2answers
217 views

The dirty coins and the three judges

All things being perfect, if we want to know if a set of coinage (all coins with the same bias) is biased, and to what degree, we would just use the outcomes and confidence intervals of the binomial ...
0
votes
0answers
17 views

Understanding Assumptions of classical Measures of reliability

Suppose we have a test comprised of $4$ items (i.e., $X_1, X_2, X_3, X_4$). We have $8$ students (i.e., $S_1, . . ., S_8$) who have taken this test. Picture below shows the test scores for these $8$ ...
0
votes
1answer
18 views

determining validity of the regression

I have performed curvilinear regression analyses on my data and the curvilinear fit is better than the linear (as determined by the F-stat of anova). I am now searching for a way to measure the ...
0
votes
1answer
55 views

Minimal detectable change based on >2 measurements

I need to calculate the minimal detectable change (MDC) for my dataset and found the following formula in several publications: $MDC = 1.96 * SD * \sqrt(1-ICC) * \sqrt2$ They argue $\sqrt2$ is used, ...
0
votes
0answers
87 views

Kappa vs Non-parametric test for intra-observer data

I am looking at a set of data which I analysed blinded (n=10) at two separate sittings. I am trying to show that there is intra-observer agreement (my measurements are repeatable and reliable). My ...
2
votes
1answer
82 views

Which ICC model to use for computer vs. human agreement

I have an agreement study with the following basic construction: ~ 400 items are rated by (1) a computer algorithm, and (2) a human rater. The ratings are on a continuous scale. I would like to ...
0
votes
0answers
18 views

Reliability of a test administered to 2 groups

In case a judgment test is administered to 2 independent groups how is it measured for test reliability: 1 group at a time or the 2 groups together?