A measure is said to have a high reliability if it produces similar results under consistent conditions. DO NOT confuse reliability with validity (see tag wiki).

learn more… | top users | synonyms

0
votes
0answers
10 views

Cronbach's alpha internal consistency test for interval scale or ratio scale data with different units

I have recently read dos Santos, S. F. & Brandi, H. S. 2015. Model framework to construct a single aggregate sustainability indicator: an application to the biodiesel supply chain. Clean ...
2
votes
2answers
30 views

Is CFA appropriate for test retest reliability?

I know CFA is often used when there is a latent variable that has measurement error that can attenuate correlations. However, does it make sense to make this same adjustment for measurement error when ...
0
votes
0answers
16 views

sum of dependent random variables [closed]

I want to find a function that determine connection between x and y that x and y both of them are determine by a common other factor. in fact x and y are dependent variables. function is like a ...
0
votes
0answers
7 views

What is a good book/online source about the subject of reliability?

I am particularly interested about Reliability function $R(t)$ Failure function $h(t)$ Renewal process Maintained systems Nonhomogenius Poisson process and reliability growth Thank you.
0
votes
0answers
14 views

Test-retest reliability

I have come across following table in a publication, where r is correlation coefficient. I could not find detailed procedures for test-retest reliability, is it any different from paired t-test ? ...
1
vote
0answers
57 views

Meta-analysis using experimental reliability data

I have a rather conceptual question. We measured experimentally the reliability of a certain readout. Now what I would like to do is to use this data to give a statement about the results of previous ...
0
votes
0answers
8 views

Determine association between/reliability of two series of frequencies

I have received two sets of data, both contain usage frequencies of different newspapers during 44 days. I don’t have any data on respondent level. The samples questioned in both datasets are ...
0
votes
1answer
25 views

How to test consistency of responses?

I am analyzing data from several trials where individuals were submitted to different stimuli and a biological response was measured. Some individuals participated in more than one trial where the ...
0
votes
0answers
8 views

Should I rely on the result of Wilcoxon Signed rank or Kruskal Wallis?

I am writing a dissertation on the relationship between age but also investigating other factors such as income, and pro-environmental behaviour. If i test household income against 'Have you taken ...
0
votes
0answers
19 views

Cronbach's Alpha if Item column shows no values

I am doing a report about Factor Analysis, but have no much experience with it. I came up with 3 factors. Factor 1 with 5 items, factor 2 with 2 items and factor 3 with 2 items. I then moved on to a ...
-1
votes
1answer
16 views

how to reliabilty test for this questionnaire

The study on academic stress of students Q1. How frequently are examinations held in your college? a) Every day b) Every week c) Once in a month d) Half -yearly e) ...
0
votes
0answers
5 views

Query on Reliability. The Cronbach's Alpha, Study on Resilience of Vulnerable Children (CYRM-28)

I am doing a research on Risk, Protective factor and Resilience of Vulnerable Children (A study based on Children residing in the Government Children's Homes in Kerala, India). I have used Child Youth ...
0
votes
0answers
14 views

software reliability modeling

Software Reliability Modelling In a given sample data set of discrete software failure data, list of test cases are given and corresponding to that test case, what are the various software bugs ...
0
votes
0answers
15 views

Weibull distribution - reliability prediction

I am using Weibull to predict hard disk failure rates. I used shape 2.0 and scale 1.2 and got fail rates to be [ 1.38888889 2.77777778 4.16666667 5.55555556 6.94444444 8.33333333 9.72222222 ...
0
votes
0answers
9 views

Negative Cronbach's alpha after reversing results of a survey

As part of my master thesis, I am conducting a PCA analysis. The analysis is aiming to find commonalities between 8 survey questions that were answered by 219 respondents. The survey was answered on a ...
1
vote
0answers
12 views

Standard Error of Measurement (SEM) in Inter-rater reliability

I am currently writing my thesis about inter-rater reliability of a diagnostic tool between raters. I want to use the standard error of measurement with the formula: SEM of rater 1 and rater 2 = SD * ...
2
votes
1answer
34 views

Correcting For Reliability/Consistency of a Measure in Meta Analysis (Cronbach's Alpha / Test Re-Test)

Is it good practice to correct for reliability of your measure in a meta analysis? This is what is suggested by Hunter and Schmidt (2004), but it seems weird to me to calculate your average effect for ...
0
votes
0answers
20 views

Categorical Principal Components model summary

I have three questions on CATPCA using SPSS. 1) If I use a varimax rotation, it gives "Model Summary" and "Model Summary Rotation". Should I report values on the Model Summary Rotation since that was ...
1
vote
1answer
33 views

Inter-rater reliability or another method?

I've asked about the mechanics of Inter-rater reliability before, but I guess I should have asked this question instead: If I have a bunch of raters and individuals (who are being judged on ...
1
vote
0answers
22 views

Does measure of reliability apply to regression coefficients?

I have a problem of estimating some kind of a test-retest reliability/intraclass correlation of the regression coefficients. I have only seen test-retest reliability apply to outcome variables, simply ...
0
votes
0answers
7 views

How many raters do I need to determine reliability of agreement on stimulus classification?

I have a stimulus set comprised of 147 videos, each with one of seven labels (hidden from raters). I want to determine how reliable label assignment is for each stimulus -- that is, I want to be able ...
0
votes
1answer
64 views

Calculation for inter-rater reliability where raters don't overlap and different number per candidate?

I want to calculate the degree to which the gymnastics judges agree on balance beam scores, i.e., "inter-rater reliability". However, not all judges judge the same candidates, and the number of judges ...
0
votes
0answers
15 views

Reliability test-retest

My Doctor asked me to send two questionnaire (one is an MCQ and the other is scale ranging from 1 strongly agree to 6 strongly disagree). He told me to send the same questionnaire to same people ...
0
votes
1answer
13 views

Reliability of the scale

Do we have to calculate Cronbach's alpha for all the questions ( items) or just the variables? I have 7 variables: 4 independent, 1 dependent, 1 mediating, and 1 moderating, and 23 items 23. ...
2
votes
1answer
71 views

How reliable is factor analysis?

I've read some papers that used factor analysis to find latent factors in data. How reliable is this technique in general for correcting identifying related factors? Are spurious results likely? Is it ...
0
votes
0answers
15 views

Intraclass correlation (reliability) for accelerometer data

I want to assess the reliability of accelerometer measurements for three days. For the calculation of the reliability I use a random-effect model (in R) to get the proportion of subject variance to ...
0
votes
0answers
17 views

Inter-rater reliability on multiple-choice tests

I have a task where coders are asked to assign categories to items by answering multiple choice questions. For each question, a coder is presented with an item and 5 categories (the total number of ...
0
votes
0answers
29 views

Reliablity and validity new questionnaire

A questionnaire with three subtopics concerning; well being, competence and empathy is adjusted and translated by us. We would like to know whether our new, shorter version of this questionnaire is ...
2
votes
1answer
48 views

(1) A flight has a 1% chance of crashing. (2) It's scheduled to fly 200 flights that week. Does (2) add any info re: your risk?

And assuming that: a) This is that particular airplane's very first flight, ever (not just for the week). b) Any distributions involved should be discrete in the sense that they should be functions ...
0
votes
0answers
20 views

Calculating reliability of a label

I have a dataset consisting of 1.1M images and 2161027 labels. (Each image can have multiple labels) I performed a validation on the subset of the dataset, with 10k images and 19067 labels. 1448 ...
2
votes
0answers
23 views

What measure of test reliability is appropriate if the final test output is not the sum of item scores?

I would like to compute a measure of reliability in the sense of Cronbach's alpha, McDonalds omega or glb. However it seems to me that all these measures assume that the final test score equates to ...
2
votes
1answer
58 views

Is empirical reliability in IRT the counterpart of internal consistency in CTT?

In IRT it is not meaningful to compute internal consistency coefficients such as Cronbachs alpha, because standard error of measurement (and thus measurement precision) is not assumed uniform across ...
1
vote
0answers
26 views

What are the most appropriate statistics to calculate using this reliability data?

10 teachers independently look at 3 randomly selected papers from a previous edition of a subject, and gives each a mark out of 100. The 3 papers will be the same across all markers. I am meant to ...
0
votes
0answers
8 views

Comparison of two measurement methods - determining measurement reliability

The aim is to compare two measurement methods. Therefore with each method two consecutive measurements were performed on 25 subjects (so overall 4 x 25 measured values) The relevant parameter is an ...
0
votes
0answers
10 views

Rubric Interrater reliability study

I'm looking at the interrater reliability of a 6-criteria grading rubric used by an online college for graduate students. I have 10 mini-studies with all four professors grading the papers of 12 ...
0
votes
0answers
8 views

reliability measurement within one scale per subject

I am trying to compute an internal reliability measurement for each subject on one type of trial within an eyetracking study. Basically I would like to know if each person is responding reliably (in a ...
2
votes
2answers
54 views

Grad level books for survivability analysis applied to manufacturing

I am looking for a grad level book on survivability analysis applied to manufacturing such as assembly lines. Preferably in R. I am familiar with measure theory, probability theory and R.
0
votes
0answers
28 views

Best way to represent survival data

I have a question regarding the best way to represent survival data to predict the reliability of bike parts. I have the manufacturing date of each bike. At the manufacturing data the bike is ...
0
votes
0answers
12 views

How to estimate the occurrence time of an event for scheduling purposes precisely?

I working on a system in which a number of requests arrive in random time. Consider the following timeline: In the above example, we have a certain number of components: ...
0
votes
0answers
38 views

Is reliability a requirement for ANOVA or regression analysis?

I would like to conduct ANOVAs and regression analyses with several DV's having a low reliability (cronbach's alpha < .40). This is probably due to the low number of items (2-3) in each scale. Is ...
0
votes
0answers
14 views

Cronbach's Alpha in an exam.

An exam has 15 x 8 minute stations where candidates are marked by an examiner while performing different tasks. The passmark set using borderline regression method + 1 Standard Deviation. Cronbach's ...
4
votes
1answer
92 views

What is Cronbach's Alpha intuitively?

I'm trying to understand Cronbach's Alpha intuitively. What is the general idea behind this construct? What properties were they trying to ensure that it had?
0
votes
1answer
63 views

Confirm correct process to use Weibull regression for reliability analysis

I am new to the field of survival analysis. I am trying to fit a Weibull distribution to predict the reliability of a part. I have information from machines that have that part. On some machines the ...
0
votes
1answer
43 views

Generate data from Dirichlet distribution? [closed]

I want to generate Dirichlet distributed data, but I don't know how to do it. Could you please help me?
0
votes
0answers
42 views

How do I use a large amount of data to inform a failure probability analysis?

I'm trying to calculate a failure probability for a given $n$-dimensional model with a large data set. I've chosen FORM/SORM as tools, which require that I calculate a joint probability distribution ...
0
votes
0answers
7 views

Correcting for multiple data points

I am validating a new measure of therapist competency in psychology. I am getting 4 clients who have taken part in 40 sessions (10 each) to answer a questionnaires about their therapy sessions ...
0
votes
1answer
17 views

Inter-rater reliability: two raters rate dataset A and two raters rate dataset B how to combine it into one number?

Me and one other person (Alice) counted the amount of boys and girls playing on a school playground (consent of parents and school were given of it being videotaped). Two other persons (Bob and ...
2
votes
0answers
42 views

Test-retest correlation for panel data

I've run an experiment in which subjects rate how much they like six different objects on a 1-5 scale on two occasions. I'd like to obtain a summary measure of how consistent are the subjects in their ...
0
votes
1answer
116 views

Understanding of the chi square statistic

I have a formula for approximating the max the mean time between failures (MTBF) that is: r / operating time (*note-MTBF is 1/ failure rate) where r is the number of failures observed. I have ...
0
votes
0answers
22 views

Correct approach for computing interrater reliability

everybody. Can't find a strong answer to this "special case" elsewhere on the interwebs, so I'm hoping you can help. I want to computer interrater agreement (actually, in this case, inter-method ...