A measure is said to have a high reliability if it produces similar results under consistent conditions. DO NOT confuse reliability with validity (see tag wiki).

learn more… | top users | synonyms

0
votes
0answers
6 views

reliability measurement within one scale per subject

I am trying to compute an internal reliability measurement for each subject on one type of trial within an eyetracking study. Basically I would like to know if each person is responding reliably (in a ...
2
votes
2answers
49 views

Grad level books for survivability analysis applied to manufacturing

I am looking for a grad level book on survivability analysis applied to manufacturing such as assembly lines. Preferably in R. I am familiar with measure theory, probability theory and R.
0
votes
0answers
11 views

how to find the mean from this question [closed]

A railway train is fitted with three engine/transmission units that can be assumed to exhibit a constant hazard with a mean life of 200 h. In a 15 h working day, calculate the probability of a train ...
0
votes
0answers
14 views

Best way to represent survival data

I have a question regarding the best way to represent survival data to predict the reliability of bike parts. I have the manufacturing date of each bike. At the manufacturing data the bike is ...
0
votes
0answers
7 views

How to estimate the occurrence time of an event for scheduling purposes precisely?

I working on a system in which a number of requests arrive in random time. Consider the following timeline: In the above example, we have a certain number of components: ...
0
votes
0answers
12 views

Is reliability a requirement for ANOVA or regression analysis?

I would like to conduct ANOVAs and regression analyses with several DV's having a low reliability (cronbach's alpha < .40). This is probably due to the low number of items (2-3) in each scale. Is ...
0
votes
0answers
8 views

Cronbach's Alpha in an exam.

An exam has 15 x 8 minute stations where candidates are marked by an examiner while performing different tasks. The passmark set using borderline regression method + 1 Standard Deviation. Cronbach's ...
4
votes
1answer
45 views

What is Cronbach's Alpha intuitively?

I'm trying to understand Cronbach's Alpha intuitively. What is the general idea behind this construct? What properties were they trying to ensure that it had?
0
votes
0answers
30 views

Confirm correct process to use Weibull regression for reliability analysis

I am new to the field of survival analysis. I am trying to fit a Weibull distribution to predict the reliability of a part. I have information from machines that have that part. On some machines the ...
0
votes
1answer
39 views

Generate data from Dirichlet distribution? [closed]

I want to generate Dirichlet distributed data, but I don't know how to do it. Could you please help me?
0
votes
0answers
19 views

How do I use a large amount of data to inform a failure probability analysis?

I'm trying to calculate a failure probability for a given $n$-dimensional model with a large data set. I've chosen FORM/SORM as tools, which require that I calculate a joint probability distribution ...
0
votes
0answers
7 views

Correcting for multiple data points

I am validating a new measure of therapist competency in psychology. I am getting 4 clients who have taken part in 40 sessions (10 each) to answer a questionnaires about their therapy sessions ...
0
votes
1answer
10 views

Inter-rater reliability: two raters rate dataset A and two raters rate dataset B how to combine it into one number?

Me and one other person (Alice) counted the amount of boys and girls playing on a school playground (consent of parents and school were given of it being videotaped). Two other persons (Bob and ...
2
votes
0answers
36 views

Test-retest correlation for panel data

I've run an experiment in which subjects rate how much they like six different objects on a 1-5 scale on two occasions. I'd like to obtain a summary measure of how consistent are the subjects in their ...
0
votes
1answer
68 views

Understanding of the chi square statistic

I have a formula for approximating the max the mean time between failures (MTBF) that is: r / operating time (*note-MTBF is 1/ failure rate) where r is the number of failures observed. I have ...
0
votes
0answers
19 views

Correct approach for computing interrater reliability

everybody. Can't find a strong answer to this "special case" elsewhere on the interwebs, so I'm hoping you can help. I want to computer interrater agreement (actually, in this case, inter-method ...
1
vote
1answer
31 views

acceptable ICC for 6-point Likert scale

I have pairs of pictures of participants making facial expressions (240 pairs per participant, 11 participants). Two raters agreed on a criterion and rated each one of the pairs on a scale from 1 to ...
0
votes
0answers
16 views

Assessing Repeatability: Random and Systematic Error

I am analyzing a study where my goal is to assess the quality (precision) of several machines. I have just one true value (measured with the goldstandard) and made 30 runs for every single machine. ...
0
votes
0answers
11 views

Fleiss Kappa: am I using it correctly?

I have a dataset where 20 subjects rated 16 audio samples according to some perceptual features on 1-9 scale and I am interested to measure agreement between raters. What I do is following: for each ...
1
vote
1answer
33 views

Lifetime estimate of components where only a fraction of them have failed so far?

I am attempting to calculate how long a component is expected to last and ultimately how many working components will be left at a future point in time. 800 have failed and 2200 are still working. I ...
1
vote
0answers
27 views

What are the most important components of an item analysis of a MCQ exam?

I'm analyzing results of undergraduate exams that range from 60 to 100 multiple choice questions. The number of students doing any given exam ranges from ~70 to ~1000. In all cases students select ...
2
votes
2answers
37 views

Quantifying consistency between two groups of raters

I have two groups of people, group A (62 people) and group B (8 people), and each person in each group responded to a test with the same 8 questions where the answer to each question was one of the ...
1
vote
0answers
19 views

Bootstrapping on Cronbachs Alpha? - Is it a good idea?

One can calculate a BCa-CI of Cronbachs Alpha, e.g. using the {psy}-Package. Is this generally a good idea? You can read here about a simulation study in this matter. ...
1
vote
0answers
16 views

Test retest reliability for a test taken multiple times

In our study, we would like to assess the reliability of a test measuring pitch discrimination thresholds. For this, we let participants take an adaptive threshold test (staircase procedure). We ...
0
votes
0answers
18 views

What method(s) exist(s) for the classification and reliability (Cohen's kappa) analysis of partial data?

I'm trying to set up a study in which I have many (about 700) images to classify. I need to measure expert judgements on this data set. Based on literature on the same subject, I am setting up a study ...
0
votes
0answers
30 views

Cronbach's Alpha

which types of items (Ordinal, nominal, scale) included in Cronbach alpha calculations? In other word, in my questionnaire i have a different types of questions (yes or no, Likert scale, and opened ...
0
votes
1answer
55 views

Finding the Maximum Likelihood Estimate in R [closed]

How do I estimate the parameters through MLE for a Likelihood function like. Note that the function has 2 multiply signs and there is a integration term in the second one.
0
votes
0answers
68 views

Interpreting summary of ICC results in R package “psych”

I have a dataframe "all.st.icc", where 27 subjects have measures of social behavior for each of 8 different years. First four rows of data are as follows: ...
8
votes
1answer
113 views

Why is backward elimination justified when doing multiple regression?

Does it not result in over-fitting? Would my results be more reliable if I added a jack-knife or bootstrap procedure as a part of the analysis?
0
votes
1answer
52 views

Reliability test in Likert scale

I have found a Likert scale measure in an article for global job satisfaction measure that included three items as follows; All in all I am satisfied with my job, In general, I don't like my job ...
3
votes
2answers
236 views

Cronbach's Alpha in Pilot Study: do I need to eliminate items if alpha is already > 0.7

In the Pilot Study, I am trying to confirm reliability of my scales: I'm performing Cronbach’s alpha test on 5 items that represent one construct of my research model (IV). I'm receiving α = .710 ...
1
vote
0answers
32 views

Test retest reliability

I tested subjects and repeated the test 2 times, so I did it 3 times in total. My question is: does anyone know how to evaluate the test-retest reliability? Is there another method (besides ...
0
votes
0answers
20 views

Which reliability index (if any) is appropriate?

Subjects were asked to endorse whether or not they have experienced various negative events (yes/no). I want to use a measure that is a sum of how many events they have experienced. Since this measure ...
0
votes
0answers
24 views

Rejecting subjects based on retest analysis

Are there objective criteria for rejecting subjects based on a test-retest analysis? Perceptual data on a discrete scale [1,...,9] for a number of users over groups of audio excerpts was collected. ...
1
vote
1answer
19 views

Split-half when intermediate values are equal

I am trying to perform a split-half for strategy use, however, as you can see the middle most users have the same score. What is the best approach to perform a split-half in this situation? Do you ...
2
votes
0answers
21 views

How to test the inter-rater reliability of a nominal coding process that generates ordinal data?

A researcher studying hunger on a tropical island asks respondents, "What did you eat for lunch yesterday?" and the possible choices are "(4) Nothing; (3) a banana; (2) an unripe coconut; (1) a ripe ...
0
votes
0answers
11 views

Conceptual difference between ordinally reliable and nominally reliable inter-rater statistic, K-alpha?

Using Krippendorff's alpha allows one to calculate an inter-rater reliability statistic on nominal, ordinal, interval, ratio, and other types of data. Assume that Krippendorff's alpha is 0.6 when ...
0
votes
0answers
26 views

Composite reliability in CFA: how to treat the negative factor loadings?

I just have a simple, stupid question. For the composite reliability calculation, we start with summing all standardized coefficients for each factor. What if the signs of some of them are negative? ...
0
votes
0answers
32 views

how to estimate scale variance only using item sd's and scale reliability

The problem I encounter is that I need to estimate the scale variance of a psychometric scale without having the original data. The only information I have is the means of the (binary) items, their ...
1
vote
1answer
27 views

Where I can find resources to learn how to calculate the sample size representativeness, and realiability and validity of questionnaires?

I'm totally newbie in psychometrics but starting from a research paper with full data I would like to understand: how to calculate if the sample size is representative; how to calculate reliability ...
1
vote
0answers
17 views

Inter-rater reliability for $k$ raters with $0$ to $n$ of $n$ categories applying to each subject

I have a data set for which I would like to calculate the inter-rater reliability. However, this data set does not seem to fit the typical models that conventional algorithms allow for. My data set ...
1
vote
0answers
14 views

Kendall's W for random subjects (not raters)

I have 5 raters (A,B,C,D,E) who evaluated 60 images on 1 to 10 points scale. The images are split into two groups: half of them were easy, half of them vere difficult (1 is easy, 10 is difficult). I ...
1
vote
1answer
27 views

An invalid questionnare with valid resultant scores?

In what situation, a questionnaire could be invalid but its resultant scores could be valid? I received a comment from a journal article reviewer that says: In keeping with the preferred ...
2
votes
2answers
60 views

Should I use paired t-tests or ANOVA when comparing multiple variables from 2 tests?

I had 50 subjects perform two 30-minute cycling time trials on separate days. I would like to compare the means of several variables from each of the two tests: Cadence, Heart Rate, Power Output, and ...
0
votes
0answers
19 views

Negative Reliability for Pre Assessment BUT Okay Reliability for Post Assessment

I have gathered data from only 6 participants participating in a program. Before running my analysis, I wanted to check my reliabilities. The 2 item PRE assessment has a negative alpha. However, when ...
0
votes
0answers
275 views

sample size for test-retest reliability

I am interested in estimating the following: test-retest reliability for ordinal items; test-retest reliability for interval-level scale scores consisting of the items; and the optimal sample size ...
0
votes
0answers
20 views

How many number of items should i include in my test?

I have designed two post test ,a recognition test and a production test.I have inserted 18 items in each test. As I sifted through articles , I 've read that we should include at least 30 items to get ...
0
votes
0answers
89 views

Examining Validity and Reliability of a new questionnaire (Likert scale)

I need to examine the psychometric properties of a newly developed questionnaire. There are 22 items with a 5-point Likert Scale. The designed questionnaire includes two main scales. How do I ...
0
votes
0answers
24 views

The Effect of Unreliable Observed Variables in Factor Analysis (FA)

My question relates to the impact of unreliable observed variables in common factor analysis. Does common factor analysis treat the unreliable portion of a variable's variance as a unique variance? ...
0
votes
1answer
188 views

What is the statistic to calculate Cronbach's alpha for Polytomous items in R

Does anyone know, what is the function which can calculate Cronbach's alpha for polytomous items with different categories in R? Thank you