An instrument is valid if it measures what it purports to measure. This is different from reliability, which means that an instrument measures (whatever it measures) in a way that is repeatable. DO NOT use this tag for `validation` model purposes.

learn more… | top users | synonyms

0
votes
0answers
5 views

convergent and discriminant validity test

Is it correct to do convergent and discriminant validity test (factor analysis, KMO, Pattern Matrix) of my research instrument only on independent variables? I use TAM in my study and found, that it ...
0
votes
0answers
17 views

Any other way to report convergent and discriminant validity of the instrument?

After Cronbach test on data received from answering to questionnaire that is available under the link: ...
0
votes
0answers
5 views

Looking for a “(numerical) reliability index” for computed Bhattacharyya distances

The Bhattacharyya distance $D_B$ between two distributions $p_1$, $p_2$ is given by $$ D_B(p_1, p_2) = \frac{1}{8} (\mu_1 - \mu_2)^T \Sigma^{-1} (\mu_1 - \mu_2) + \frac{1}{2}\log\frac{\det ...
3
votes
2answers
59 views

Cronbach's Alpha in Pilot Study: do I need to eliminate items if alpha is already > 0.7

In the Pilot Study, I am trying to confirm reliability of my scales: I'm performing Cronbach’s alpha test on 5 items that represent one construct of my research model (IV). I'm receiving α = .710 ...
0
votes
0answers
13 views

The differences between criterion validity and operational validity

In some psychological articles, I have seen the term "operational validity". It is often used instead of "criterion validity". What are the differences between these two concepts?
2
votes
0answers
5 views

How to test the inter-rater reliability of a nominal coding process that generates ordinal data?

A researcher studying hunger on a tropical island asks respondents, "What did you eat for lunch yesterday?" and the possible choices are "(4) Nothing; (3) a banana; (2) an unripe coconut; (1) a ripe ...
1
vote
1answer
20 views

Where I can find resources to learn how to calculate the sample size representativeness, and realiability and validity of questionnaires?

I'm totally newbie in psychometrics but starting from a research paper with full data I would like to understand: how to calculate if the sample size is representative; how to calculate reliability ...
1
vote
1answer
25 views

An invalid questionnare with valid resultant scores?

In what situation, a questionnaire could be invalid but its resultant scores could be valid? I received a comment from a journal article reviewer that says: In keeping with the preferred ...
0
votes
0answers
20 views

Is a representative sample necessary to assess construct validity?

To generalize results to a greater population, a representative sample is necessary. In developing a psychometrically valid instrument, via confirmatory factor analysis or item response theory, we are ...
0
votes
0answers
54 views

Examining Validity and Reliability of a new questionnaire (Likert scale)

I need to examine the psychometric properties of a newly developed questionnaire. There are 22 items with a 5-point Likert Scale. The designed questionnaire includes two main scales. How do I ...
3
votes
0answers
63 views

How high is too high with Cronbach's alpha?

Someone in my organization just sent me a scale they developed (modified items from five separate scales used in published research) to measure an employee's readiness for change. They tested the ...
4
votes
1answer
28 views

Is this method of arriving at a statistic valid?

The following is an excerpt from an article about gender bias : Overall, 34 per cent of respondents readily admitted that their family preferred having more boys. Given the sensitive nature of ...
0
votes
0answers
21 views

Validity in IRT Style

From my reading, I found that IRT generally provides more sophisticated information. Therefore, in regards of unidimensional 3PL IRT model, how should I report the finding about validity in ...
0
votes
0answers
33 views

How can you judge the statistical confidence and validity of output from a multi arm bandit algorithm like UCB1

To say something about the validity of outcomes in frequentist statistics we have concepts like significance levels and statistical power and in Bayesian analytics we have credible intervals. In a ...
5
votes
2answers
32 views

Validity of combining surveys and sociodemographics to estimate miles walked in census tracts

I ran across this paper that describes an interesting approach to combining survey and sociodemographic data: Salon. Estimating Total Miles Walked and Biked by Census Tract in California. State of ...
0
votes
1answer
73 views

Is 0.40 Alpha score (KR-20) acceptable for a true-false test?

I have 10 true-false-Idont know test items. I codded correct responses as 1, incorrect ones and dont knows as 0. I got 0.40 alpha score (or KR-20). Do you think that this score is acceptable for such ...
0
votes
0answers
43 views

EFA or CFA for validity

I need to test validity of measurement scales in my survey. 1) Is it true that both can be used to test measurement validity? 2) Or there are some types of validity that can be tested with EFA and ...
0
votes
0answers
37 views

Discriminant validity testing

I am required to assess the discriminant validity of the constructs in my model. I have used the Fornell and Larcker (1982) method, however I now need to explore discriminant validity using the ...
0
votes
0answers
9 views

What conclusion can be reached when failing at validating a scale?

I developed a personality test which resulted with a high internal consistency but without factorial validity (construct). I know that the first conclusion which can be drawn is that the test was ...
1
vote
0answers
106 views

How to measure construct validity?

I recently received feedback from a journal (in education) that "The idea that you can assess construct validity through a factor analysis is inconsistent with how we usually think about validity in ...
1
vote
0answers
24 views

Factor analysis without theoretical congruence

I've been working with certain personality theory, which I'm trying to measure in a sample of students. It's a foreign theory, so I created an instrument for my specific population. I got a Cronbach's ...
1
vote
0answers
18 views

Choosing between two scales

I am investigating a scale of consumer ethnocentrism - CETSCALE. I do not have much knowledge in econometrics, scales, etc., however: a) research papers report that the standard 17-item scale is ...
0
votes
0answers
45 views

Statistical significance of cluster validity

Hi I m working on a unsupervised problem to partition my dataset. I have access to the class labels for this dataset. Now I am trying to use Jaccard coefficient to compute correlation between cluster ...
0
votes
0answers
22 views

Calculate Cronbach's alpha over test set or over test time?

Reviewers have requested me to calculate Cronbach's alpha for tests I administered during an experiment. There are two ways I can do it (at least), and I'm wondering which is more appropriate. ...
0
votes
0answers
28 views

Validating a manual classification

I am classifying some political articles by their bias w.r.t. a particular issue into 5 categories (strongly in favour, mildly in favour, neutral, mildly against, strongly against). For most cases, ...
0
votes
0answers
23 views

Validity of running 2 conditional logistic regression tests on a data set

My current research is about hibernation site selection in box turtles. In order to determine selection, I collected a range of variables about the soil that the turtle hibernated in as well as the ...
2
votes
1answer
58 views

households survey data - determine 'valid' product categories

I'm working with household survey data where households report their expenditure for various products. The goal is to look into income elasticities for various products thus in essence I'm interested ...
0
votes
0answers
208 views

Covariance between factors in CFA (Amos)

I ran a Confirmatory Factor Analysis (CFA) in Amos and obtained a model with high covariance between the factors. James described this covariance in his video as alarming. What can I do to tackle ...
1
vote
1answer
128 views

Evaluation measures of overlapping clustering

I have a dataset of Facebook users and a set of different clustering algorithms. The project goal is to draw up a rank between these algorithms in order to understand which of them are the good ones. ...
1
vote
0answers
61 views

Validity of a statistical procedure to calculate an odds ratio for a Fisher's exact test?

I am trying to determine whether this is a valid statistical procedure. I have ~1000 samples, and each sample has around 1-500 data points. Each data point consists of two fractions (value1 and ...
0
votes
0answers
43 views

Selection Bias with Groups

I have conducted a pre-post test study with a set of 20 participants. There were two independent variables and therefore 4 groups of 5 participants each. The experiment took place in a classroom and ...
11
votes
2answers
318 views

Identification of useless questions from a questionnaire

I'm developing a questionnaire. To improve its reliability and validity I want to use statistical methods. I want to eliminate questions whose answers are always the same. This means that nearly all ...
4
votes
4answers
783 views

If a measure is valid (but not necesarily reliable), can it be consistently replicated?

I had this question on an exam, and I was positive that the answer was A. The Beck Depression Inventory is a scale intended to measure depression levels, with higher scores indicative of higher ...
2
votes
0answers
42 views

Small data set and validity of Multilevel Regression results

I want to estimate a hierarchical multilevel regression (two levels). My data set is fairly small: 25 groups and 255 individual observations distributed unequally among groups (group member size ...
0
votes
2answers
109 views

Do reliability and construct validity have same technical(statistical) assumptions?

It appears that reliability and construct validity have a similar backing in terms of statistics. Is it true ?
2
votes
1answer
375 views

CFA Fit Measure Thresholds for New Scales

A well established questionnaire measuring interpersonal relations is modified to measure interorganizational relations. 11 parameters of the relation are determined by 4 items each. Items are removed ...
0
votes
1answer
137 views

Can we use CFA (confirmatory factor analysis) to ascertain validity coefficient?

Validity coefficient is an important measure for test development especially in psychology and sociology wherein we have too indirect measurements. Perhaps we have different ways to know its size.
-2
votes
2answers
564 views

What is the difference between reliability and validity?

A test should be designed in a manner to get valid information. Is there any relevance of reliability in constructing a test?.
3
votes
2answers
345 views

How does reliability and validity affect the results (descriptive statistics)?

Okay, I think this might be a stupid question but I just can't quite seem to get there. I understand what reliability and validity is, but I don't understand just what I should do with it when looking ...
3
votes
1answer
1k views

Validity Index Pseudo F for K-Means Clustering

The Validity Index "Pseudo F" is described as: (between-cluster-sum-of-squares / (c-1)) / (within-cluster-sum-of-squares / (n-c)) with c beeing the number of clusters and n beeing the number of ...
0
votes
0answers
64 views

Is this a valid test?

I have 5000 cases vs 5000 controls with positive/negative outcome. chi.test shows no significance for this data. I get significance when subsetting cases on VarX, ...
1
vote
1answer
442 views

Doing CFA on a known theoretical model, but having problems with convergent and discriminant validity

I did a survey based on the UTAUT model / theory with standard questions used. The N was 150. I am trying to validate the model for my survey with structural equation modeling (via the AMOS software). ...
1
vote
0answers
35 views

how to compare partial ratings of a set of data by different groups of participants?

So lets say I have N items and I want to rate their relevance (say on a 5-point likert scale: 1 being least and 5 being most relevant) to the "perception of beauty" and I have M number of ...
-2
votes
1answer
82 views

What is validity of an inference procedure?

What is validity of an inference procedure? The inference procedure can be point estimator, confidence interval, or a testing rule. I saw "validity" in Diggle et al's Longitudinal Data Analysis: ...
5
votes
1answer
367 views

How can reliability and validity of content analysis be quantified when there is only one person coding the data?

How can reliability and validity of content analysis be quantified when there is only one person coding the data?
-2
votes
1answer
51 views

Hand hygiene observations

How much validity is there in doing external hand hygiene observations in a hospital where another facility observes for a total of 6 hrs. twice a year and reports that data out, when the observed ...
1
vote
1answer
387 views

Using existing questionnaire

I will be doing my colloquium and will be using existing questionnaires. I would like to know if there are authors/authorities who say that an existing questionnaire need not be validated. The ...
1
vote
1answer
16k views

How to calculate validity of questionnaire in SPSS?

I have 3 types of questionnaires, A 15 item scale with each item on a 7-point Likert type scale. Six subscales each containing different number of items 21 subscales each containing 4 items. ...
1
vote
0answers
195 views

How to test relative validity statistically?

I am trying to understand a) what relative validity is, and b) how to test it statistically. So far I have gathered, from sources such as this that relative validity is a larger reliability in ...
1
vote
2answers
189 views

Under what circumstances is an N < 30 acceptable?

I am doing some research in which I want to expose subjects to an "unusual" event and record their response. I am recording the subjects' responses via a sensitive brain instrument, so I've been told ...