An instrument is valid if it measures what it purports to measure. This is different from reliability, which means that an instrument measures (whatever it measures) in a way that is repeatable. DO NOT use this tag for `validation` model purposes.

learn more… | top users | synonyms

1
vote
0answers
19 views

Validity of a statistical procedure to calculate an odds ratio for a Fisher's exact test?

I am trying to determine whether this is a valid statistical procedure. I have ~1000 samples, and each sample has around 1-500 data points. Each data point consists of two fractions (value1 and ...
0
votes
0answers
23 views

Selection Bias with Groups

I have conducted a pre-post test study with a set of 20 participants. There were two independent variables and therefore 4 groups of 5 participants each. The experiment took place in a classroom and ...
10
votes
2answers
187 views

Identification of useless questions from a questionnaire

I'm developing a questionnaire. To improve its reliability and validity I want to use statistical methods. I want to eliminate questions whose answers are always the same. This means that nearly all ...
4
votes
4answers
148 views

If a measure is valid (but not necesarily reliable), can it be consistently replicated?

I had this question on an exam, and I was positive that the answer was A. The Beck Depression Inventory is a scale intended to measure depression levels, with higher scores indicative of higher ...
0
votes
0answers
47 views

Assessing questionnaire reliability using test-retest measures

I am an undergraduate student with not much knowledge regarding statistics. I was hoping you would be able to help me by answering a question. I am currently checking the reliability and validity of ...
2
votes
0answers
26 views

Small data set and validity of Multilevel Regression results

I want to estimate a hierarchical multilevel regression (two levels). My data set is fairly small: 25 groups and 255 individual observations distributed unequally among groups (group member size ...
0
votes
2answers
76 views

Do reliability and construct validity have same technical(statistical) assumptions?

It appears that reliability and construct validity have a similar backing in terms of statistics. Is it true ?
1
vote
1answer
111 views

CFA Fit Measure Thresholds for New Scales

A well established questionnaire measuring interpersonal relations is modified to measure interorganizational relations. 11 parameters of the relation are determined by 4 items each. Items are removed ...
0
votes
1answer
44 views

Can we use CFA (confirmatory factor analysis) to ascertain validity coefficient?

Validity coefficient is an important measure for test development especially in psychology and sociology wherein we have too indirect measurements. Perhaps we have different ways to know its size.
3
votes
1answer
117 views

How does reliability and validity affect the results (descriptive statistics)?

Okay, I think this might be a stupid question but I just can't quite seem to get there. I understand what reliability and validity is, but I don't understand just what I should do with it when looking ...
2
votes
1answer
86 views

Validity Index Pseudo F for K-Means Clustering

The Validity Index "Pseudo F" is described as: (between-cluster-sum-of-squares / (c-1)) / (within-cluster-sum-of-squares / (n-c)) with c beeing the number of clusters and n beeing the number of ...
0
votes
0answers
26 views

Does true correlation in statistics represent validity coefficent as used in psychometrics? [duplicate]

The statistics appears to have borrowed certain concepts and models from psychometric theories and models. Is it a correct to presume that the two terms are equivalent?
0
votes
0answers
50 views

Is this a valid test?

I have 5000 cases vs 5000 controls with positive/negative outcome. chi.test shows no significance for this data. I get significance when subsetting cases on VarX, ...
1
vote
0answers
180 views

Doing CFA on a known theoretical model, but having problems with convergent and discriminant validity

I did a survey based on the UTAUT model / theory with standard questions used. The N was 150. I am trying to validate the model for my survey with structural equation modeling (via the AMOS software). ...
1
vote
0answers
23 views

how to compare partial ratings of a set of data by different groups of participants?

So lets say I have N items and I want to rate their relevance (say on a 5-point likert scale: 1 being least and 5 being most relevant) to the "perception of beauty" and I have M number of ...
-2
votes
1answer
50 views

What is validity of an inference procedure?

What is validity of an inference procedure? The inference procedure can be point estimator, confidence interval, or a testing rule. I saw "validity" in Diggle et al's Longitudinal Data Analysis: ...
5
votes
1answer
155 views

How can reliability and validity of content analysis be quantified when there is only one person coding the data?

How can reliability and validity of content analysis be quantified when there is only one person coding the data?
-2
votes
1answer
50 views

Hand hygiene observations

How much validity is there in doing external hand hygiene observations in a hospital where another facility observes for a total of 6 hrs. twice a year and reports that data out, when the observed ...
1
vote
1answer
134 views

Using existing questionnaire

I will be doing my colloquium and will be using existing questionnaires. I would like to know if there are authors/authorities who say that an existing questionnaire need not be validated. The ...
1
vote
1answer
5k views

How to calculate validity of questionnaire in SPSS?

I have 3 types of questionnaires, A 15 item scale with each item on a 7-point Likert type scale. Six subscales each containing different number of items 21 subscales each containing 4 items. ...
0
votes
0answers
103 views

Chi-square difference test for assessing discriminant validity

I would like to understand about the concepts and methods of chi-square difference test to test discriminant validity for measurement model in SEM. Pleas tell me and you can raise the examples that ...
1
vote
0answers
102 views

How to test relative validity statistically?

I am trying to understand a) what relative validity is, and b) how to test it statistically. So far I have gathered, from sources such as this that relative validity is a larger reliability in ...
1
vote
2answers
170 views

Under what circumstances is an N < 30 acceptable?

I am doing some research in which I want to expose subjects to an "unusual" event and record their response. I am recording the subjects' responses via a sensitive brain instrument, so I've been told ...
1
vote
0answers
398 views

How to validate a survey when using Likert scale?

I have developed a very simple survey to measure the attitudes of students toward science fairs. It makes use of the Likert Scale and Open Ended Questions. Do you have a recommendation for the type ...
8
votes
2answers
2k views

How to reduce number of items using factor analysis, internal consistency, and item response theory in conjunction?

I am in the process of empirically developing a questionnaire and I will be using arbitrary numbers in this example to illustrate. For context, I am developing a psychological questionnaire aimed at ...
1
vote
0answers
159 views

validity coefficient & $\rho$

Is the validity coefficient the same thing as $\rho$, where $\rho = \frac{COV_{XY}}{SD_X SD_Y}$?
4
votes
3answers
303 views

Looking for a good online resource about survey design and analysis

What is a good and relatively brief overview of good practice for getting valid results from surveys. I'm particularly interested in something about good survey design and analysis.
5
votes
3answers
899 views

Can I delete excessive number of multivariate outliers, like over 10% in sample?

I have a dataset with around 9000 cases, I'm running a factor analysis and I have found that 1100 cases are identified as being a multivariate outlier. Is it alright for me to got ahead and delete ...
3
votes
2answers
378 views

Reliability and validity of a standard survey

I had eleven people play five versions of a video game in single sessions, for a total of 55 sessions (11x5). I used the Task Evaluation Questionnaire from the Intrinsic Motivation Inventory to ...
2
votes
2answers
718 views

References for creating neuropsychological composite scores

I am a doctoral student in clinical neuropsychology, and am in the process of creating z-unit weighted composite scores based on a battery of ability tests (i.e., attention/executive functions ...