Questions tagged [validity]
An instrument is valid if it measures what it purports to measure. This is different from reliability, which means that an instrument measures (whatever it measures) in a way that is repeatable. DO NOT use this tag for `validation` model purposes.
136 questions
0
votes
0
answers
29
views
Algorithm to calculate validity of document?
I have a dataset of text-documents, which are all considered valid. Also I have a set of manually defined features that are required to be in the document for it to be considered valid. The features ...
- 82.8k
0
votes
1
answer
290
views
quantitative relation between standard error of measurement and validity
Is there a function to estimate the gain in predictive validity with the decrease of the standard error of measurement?
E.g. If the short version of a test has predictive validity of 0.35 and ...
CommunityBot
- 1
3
votes
1
answer
174
views
Teaching examples for internal validity: published experiments later shown to be confounded in an interesting way
There are some good examples of interesting/fun spurious correlations here (Examples for teaching: Correlation does not mean causation). Many/most of these are "funny" examples that aren't likely to ...
- 82.8k
1
vote
0
answers
11
views
Measuring Validity and reliability
Say you have an independent train ticket system where automatic refunds would be issued in case of a delayed arrival. To determine the arrival time of the train at a certain station you collect data ...
- 111
1
vote
1
answer
246
views
Correlation between questionnaire to test validity
I'm doing a validation of a questionnaire for my thesis and I should investigate the validity of my test with others one. To do so I've been asked to calculate the correlation between two scales and I'...
- 3,010
0
votes
0
answers
26
views
Questionnaire validity statistical significance
My colleagues and I, have developed a questionnaire of 116 questions, segmented in 7 constructs. Cronbach's a indices for each construct are above 70%, and it is content valid. However, when we apply ...
- 73
3
votes
2
answers
2k
views
How does reliability and validity affect the results (descriptive statistics)?
Okay, I think this might be a basic question but I just can't quite seem to get there. I understand what reliability and validity is, but I don't understand just what I should do with it when looking ...
- 1,622
2
votes
1
answer
73
views
Where I can find resources to learn how to calculate the sample size representativeness, and realiability and validity of questionnaires?
I'm totally newbie in psychometrics but starting from a research paper with full data I would like to understand:
how to calculate if the sample size is representative;
how to calculate reliability ...
- 1,622
0
votes
0
answers
28
views
Survey Question Validity
I have a survey that asks the user to report their debt from their undergraduate education and their debt from their graduate education. I intend to add these two averages to measure their average ...
- 334k
0
votes
0
answers
28
views
Generalizability study of subscaled rating instrument
I have a school rating ($1$ thru $4$) instrument that consists of $9$ subscales (e.g., classroom instruction, school management etc.). Under each subscale, I have $6$ items on which rating occurs.
I ...
- 4,056
0
votes
1
answer
79
views
Is it always needed to do a validity test for variables?
Let's say I have one variable called "perceived image", which is measured with 6 items (questions) that I adapted from previous literature. If I have a Cronbach's alpha > 0.7 (for reliability), do I ...
- 128k
3
votes
1
answer
300
views
Do I estimate factor loadings in a confirmatory factor anlysis (CFA) aimed at verifying an exploratory factor analysis (EFA)?
I decided to use a questionnaire published by another researcher (paper and supplementary here).
In the article they perform an EFA, find two factors, and report the resulting factor loadings (...
- 338
0
votes
0
answers
96
views
How to test nomological validity with lavaan?
in my understanding, I need to test nomological validity with SEM. But what exactly should I look at in e.g. summary(fit1, standardized=TRUE, fit.measures=TRUE)?
Code for reference:
...
- 569
2
votes
1
answer
1k
views
Missing data and internal and external validity
There are quite a number of questions about missing data, but I can't find one related to its consequences on validity in the context of regression. Please correct me if this is not the case. The ...
CommunityBot
- 1
0
votes
0
answers
844
views
Wilcoxon signed-rank test for paired and repeated measures data
I have a dataset with 45 participants, each with 1-7 datapoints/observations (126 total observations). Each of these observations represents a window of time in which two different methods attempt to ...
- 30.7k
1
vote
1
answer
39
views
Are there any studies of generalized error performance that don't assume data quality is constant with sample size?
As far as I know, much of the statistical and machine learning literature where modeling algorithms are compared for their generalization error performance as a function of sample size (think of ...
- 39.6k
0
votes
0
answers
14
views
factorial designed study within between
IV1 : alcohol use, IV2: breathing exercise (both of them within subject variable) IV3: gender (between subject variable). My research question is that: ''Does alcohol usage and breathing exercise's ...
- 82.8k
7
votes
3
answers
2k
views
Validity of using a combination of Wilcoxon tests and Spearman rho as an alternative to GLM?
I recently read a paper where the authors used either (a) a generalised linear model or (b) a combination of Wilcoxon rank sum tests and Spearman’s rank correlation, depending on whether the data ...
- 5,215
1
vote
1
answer
292
views
Calculating Confidence Interval with log-transformation
I fitted a linear regression with a 3rd order interaction and adjustment for confounding variable which looks like this:
$$
log(Y) = β_1S + β_2T + β_3M + β_4ST + β_5SM + β_6TM + β_7STM + β_XX
$$
...
- 1,430
0
votes
1
answer
32
views
Can findings based on low values be considered valid?
Lets say i do have a TimeSeries with N > 300 but values like [0, 1, 2, 1, 2, 0, 2, ...] representing the visitors count of a website per day.
Since there are only few visitors and each of them can be ...
- 35.2k
0
votes
1
answer
206
views
If the Pearson correlation coefficient is small/weak, can it have criterion validity?
The r value I have obtained is -0.190 and p = 0.410.
As scores on stress increase, scores on eating habits decrease.
I know from my textbook that criterion validity is about whether your measure is ...
- 12.9k
3
votes
1
answer
133
views
What does "the hypotheses ... are verified as null" mean?
Context:
This article examines the criterion-related validity of the results obtained by the application of four different methods of assessment to the correction of a second-year exam of
...
CommunityBot
- 1
2
votes
2
answers
607
views
Reweighting features in PAM/K-means clustering
As stated in Hennig et al. 2016 Handbook of cluster analysis:
If for subject matter reasons some variables are more important than
others regardless of the within-variable variation, one could ...
- 219
9
votes
3
answers
970
views
Does a statistic used as a measure becomes invalid after it's reported on? [closed]
Apologies if this the wrong place to ask. but i'm trying to explain to a director that his Statistics don't show the truth because he is demanding that the stats are what is reported on and staff are ...
0
votes
1
answer
166
views
How to assess predictive validity of a variable on the outcome?
I am looking to assess the predictive validity of two measures (M1 and M2) on a binary outcome. After constructing my model, do I just calculate odds ratios and compare the odds ratios for the two ...
- 2,715
12
votes
2
answers
17k
views
How to reduce number of items using factor analysis, internal consistency, and item response theory in conjunction?
I am in the process of empirically developing a questionnaire and I will be using arbitrary numbers in this example to illustrate. For context, I am developing a psychological questionnaire aimed at ...
- 5,257
1
vote
0
answers
78
views
Measuring emotion differentiation on a within-person level.
To measure emotion differentiation, which refers to the precision of an individual to accurately describe and distinguish between different emotional states, many studies have used a between-person ...
- 81
0
votes
1
answer
51
views
Validate 10% sample from population
I have two databases which include amount of passengers between two routes.
One is the full dataset, while the other is supposedly a 10% sample.
So for example, the full database will display a route ...
- 128k
2
votes
2
answers
843
views
Why does a factor need to be internally consistent?
I'm wondering exactly what the question says: Why does a factor (as linear combination of its items) need to be internally consistent?
If I want to use a factor found in a factor analysis as a ...
- 58.8k
2
votes
2
answers
11k
views
What is the difference between reliability and validity?
A test (for example, psychological) should be designed in a manner to get valid information, i.e. it must be valid instrument. Is there any relevance of reliability in constructing a test, too? What ...
- 58.8k
0
votes
0
answers
56
views
Are reliability and validity measures necessary for all parts of a survey
I am conducting analysis on a questionnaire which was a three-part study, not three separate studies. I asked the same exact set of validated scale questions for specific constructs for three ...
- 1
0
votes
1
answer
34
views
Validity / reliability of instrument used by 2 people at 2 sites
How can the validity/reliability of a semi-structured interview tool be determined, conducted by two persons at two different sites separately? For instance, one person is using the tool with male ...
- 20.6k
3
votes
1
answer
247
views
Survival analysis: Excluding cases of trivial follow-up time?
I am finishing up the data collection for my retrospective cohort study, which I have designed with the intention of conducting survival analysis. Our outcomes of interest are recurrence, metastasis, ...
- 35.2k
0
votes
0
answers
78
views
Cronbach alpha application
I developed a questionnaire to measure some factors influence in decision using related literature and after some interviews. Now I have to validate and see how to distribute among the target group. I ...
- 58.8k
0
votes
1
answer
2k
views
Should I calculate Cronbach's alpha for a linear regression model?
I'm interested in modelling a trait Y using variables X1, X2, X3.
I'm using a linear regression model to predict trait Y using the independent variables and therefore want to avoid multicollinearity ...
- 5,215
1
vote
0
answers
18
views
Do Standard errors and percentage of valid instances from same data set effects the F1 score?
Considering we have a data set of 944 elements. One with 45% valid instances (433 instances) and the other with 73.8% valid instances (697 instances). The F1 score ...
- 82.8k
3
votes
2
answers
5k
views
EFA or CFA for validity
I need to test validity of measurement scales in my survey.
1) Is it true that both can be used to test measurement validity?
2) Or there are some types of validity that can be tested with EFA and ...
- 2,235
1
vote
1
answer
2k
views
How is independent T-Test valid for unequal samples?
I have two groups that I wish to assess for statistically significant differences (n1=1520 / n2=115). I'm actually using T-Test for independent samples.
As far as I know, there isn't a problem here ...
- 7,882
3
votes
1
answer
94
views
households survey data - determine 'valid' product categories
I'm working with household survey data where households report their expenditure for various products. The goal is to look into income elasticities for various products thus in essence I'm interested ...
- 82.8k
0
votes
2
answers
2k
views
How can I determine the accuracy of correlation?
When I performed correlation analysis of C02 emission with around four-hundred attributes, i found out that C02 emission and cereal production has a correlation value of 0.98 where as the most obvious ...
- 2,121
1
vote
0
answers
23
views
Validity of conclusion of statistical tests when comparing two groups of observations in an experiment
I am just reading the results of an experiment, and I am having a hard time estimating their validity.
In the experiment, several participant had to perform a certain task twice, once with the ...
- 11
2
votes
1
answer
1k
views
Chi-square Discriminant Validity Test with Lavaan (R)?
I have a four factor scale that I just finished CFA on, and I was advised to use the chi-square test of differences to check for discriminant validity to reinforce my AVE based test for it. I read ...
- 5,566
0
votes
0
answers
37
views
Does reliability/validity apply on manually summed items?
For a subscale of mine, it showed a reliability in the 0.6 range which is in the 'questionable' range and it showed an AVE (convergent validity) of just 0.3. Now I know that it will be perceived as a ...
- 121
0
votes
1
answer
75
views
Convergent Validity with Multiple New Scales
I am interested in creating a new scale in a field that is overwhelmingly understudied in the research literature. I would like to test the convergent validity by modifying another survey instrument ...
- 3
0
votes
0
answers
31
views
Reliability/validity required for the mean of combined items?
Let's just say that the 4 items making one construct are not well correlating (for whatever reason) and their Cronbach is 0.62 despite loading into 1 factor. Is there an alternative way to proceed? ...
- 121
1
vote
0
answers
46
views
dependence among sub-questions in the essay exam
In the essay question, one main question includes several sub-questions that are ordered based on the thinking skills, from low (knowledge level) to high (analysis level).
Main question (total: 10 ...
- 141k
4
votes
4
answers
8k
views
If a measure is valid (but not necesarily reliable), can it be consistently replicated?
I had this question on an exam, and I was positive that the answer was A.
The Beck Depression Inventory is a scale intended to measure
depression levels, with higher scores indicative of higher ...
CommunityBot
- 1
3
votes
0
answers
385
views
EFA / SEM: Next steps after bad fit of measurement models
i want to check factorial and predictive validity of a 24 item questionnaire. For the factorial validity i calculated Measurement Models. Subsequently i wanted to calculate SEMs for predicitve ...
- 31
2
votes
0
answers
143
views
How to obtain statistical information from a Likert scale with a sample size smaller than the number of questions?
Research and sample properties
I'm measuring the degree of a certain concept related to marketing and sustainable development throughout an entire industry. The theoretical model is depicted next:
...
2
votes
1
answer
541
views
General validity/reliability of questionnaires
I have a general question concerning the reliability/validity of questionnaires. I'm working in Human Resources and we plan to conduct a workplace engagement survey. I attended a conference and a ...
