Questions tagged [validity]
An instrument is valid if it measures what it purports to measure. This is different from reliability, which means that an instrument measures (whatever it measures) in a way that is repeatable. DO NOT use this tag for `validation` model purposes.
108
questions
0
votes
0answers
10 views
Intertia significantly decreasing after running multiple k-mean clustering models
I have been spending some time running k-means clustering models using scikit-learn on a variety of feature combinations and have been using the inertia value to compare models to one another.
I've ...
0
votes
0answers
12 views
AVE and CR in JASP?
I'm new to the software JASP and need some help. Is it possible to calculate average variance extracted (AVE) and composite reliability (CR) in JASP? I need them in order to test validity and ...
0
votes
0answers
22 views
Comparing two CFA models in AMOS (only 4 items per 1-2 factors)
I am trying to test in AMOS the discriminant validity of a 3-item measure (items "1, 2 and 3" against a 1-item measure "item Z" (both are Likert scale questionnaire instruments), by comparing whether ...
2
votes
0answers
23 views
comparison between two measurement devices
I have two measurement devices accelerometer A and B. The measurement is the step time of a person walking in a corridor. A is the gold standard. I want to check the validity and reliability of B. how ...
0
votes
0answers
13 views
What do you do if participants of your study take a pretest, but not post test?
I gave pretests to several groups of participants ranging from 20 to 45 people, of which were all completed. When I gave out the post-tests, I only received back about half of the groups'. Also, ...
0
votes
0answers
13 views
Validity and Reliablity Multiple Choice Question Questionnaire
I gave questionnaires to 7-8-year-old children to answer questions about oral health knowledge. I've already collected the data but I'm still confused with validity and the reliability in SPSS.
I ...
0
votes
0answers
19 views
Survey Question Validity
I have a survey that asks the user to report their debt from their undergraduate education and their debt from their graduate education. I intend to add these two averages to measure their average ...
0
votes
0answers
23 views
Generalizability study of subscaled rating instrument
I have a school rating ($1$ thru $4$) instrument that consists of $9$ subscales (e.g., classroom instruction, school management etc.). Under each subscale, I have $6$ items on which rating occurs.
I ...
0
votes
0answers
12 views
Cronbachs alpha interpretation - what does var.r and med.r mean?
i have used the psych::alpha to find the cronbachs alpha values, but i cannot find any interpretation for some parts of the output, namely:
...
0
votes
1answer
22 views
Is it always needed to do a validity test for variables?
Let's say I have one variable called "perceived image", which is measured with 6 items (questions) that I adapted from previous literature. If I have a Cronbach's alpha > 0.7 (for reliability), do I ...
2
votes
1answer
43 views
Do I estimate factor loadings in a confirmatory factor anlysis (CFA) aimed at verifying an exploratory factor analysis (EFA)?
I decided to use a questionnaire published by another researcher (paper and supplementary here).
In the article they perform an EFA, find two factors, and report the resulting factor loadings (...
0
votes
0answers
5 views
Removing cross-loading items to obtain discriminant validity
I am using theory of planned behavior for my research. the Dependent variable is Entrepreneurial intention.
I had trouble with obtaining discriminant validity for I stumbled upon a youtube video ...
0
votes
0answers
29 views
How to test nomological validity with lavaan?
in my understanding, I need to test nomological validity with SEM. But what exactly should I look at in e.g. summary(fit1, standardized=TRUE, fit.measures=TRUE)?
Code for reference:
...
0
votes
0answers
11 views
Convergent & Discriminant Validity -Same construct (measured by different meaures) in two different samples
I don't know how to go about looking at the Convergent & Discriminant Validity of the same construct measured with two different scales in two different contexts.
This is a cross-cultural ...
0
votes
1answer
32 views
Validity of ARDL regression when there is I(2) variable
I am trying to model relationship between Y(t) and Y(t-1), X(t), X(t-1) and Z(t) using ARDL model. Most cases these time-series are I(0) or I(1). However sometimes I encounter I(2) or higher order.
...
0
votes
0answers
11 views
Is pooling acceptable to evaluate information extraction?
When dealing with information extraction of unbalanced classes (e.g. the desired class has a prevalence of 0.5%), the required sample size for validation might be huge (thousands of cases and more), ...
0
votes
1answer
558 views
item-total correlation vs. inter-item correlation
I am assesing the reliability and validity of an 16-item multuiple choice test supposed to test language skills. Cronbach's apha is "questionable" with .65. I am having troubles interpreting the ...
0
votes
0answers
27 views
What are the validity test of the Maximum likelihood estimation?
I know that the OLS has the standard validity test for assumption violation but when it come to the MLE how can we tell that the coefficients are valid for the conclusion
0
votes
0answers
102 views
Wilcoxon signed-rank test for paired and repeated measures data
I have a dataset with 45 participants, each with 1-7 datapoints/observations (126 total observations). Each of these observations represents a window of time in which two different methods attempt to ...
3
votes
0answers
54 views
How can statistics be used to avoid “Lending False Credibility To Decisions We've Already Made”
In light of this article Data Science Has Become About Lending False Credibility To Decisions We've Already Made published in Forbes, I would appreciate input from the statistical and data science ...
1
vote
1answer
15 views
Are there any studies of generalized error performance that don't assume data quality is constant with sample size?
As far as I know, much of the statistical and machine learning literature where modeling algorithms are compared for their generalization error performance as a function of sample size (think of ...
0
votes
0answers
12 views
factorial designed study within between
IV1 : alcohol use, IV2: breathing exercise (both of them within subject variable) IV3: gender (between subject variable). My research question is that: ''Does alcohol usage and breathing exercise's ...
0
votes
1answer
92 views
Calculating Confidence Interval with log-transformation
I fitted a linear regression with a 3rd order interaction and adjustment for confounding variable which looks like this:
$$
log(Y) = β_1S + β_2T + β_3M + β_4ST + β_5SM + β_6TM + β_7STM + β_XX
$$
...
0
votes
1answer
17 views
Can findings based on low values be considered valid?
Lets say i do have a TimeSeries with N > 300 but values like [0, 1, 2, 1, 2, 0, 2, ...] representing the visitors count of a website per day.
Since there are only few visitors and each of them can be ...
2
votes
1answer
92 views
What does “the hypotheses … are verified as null” mean?
Context:
This article examines the criterion-related validity of the results obtained by the application of four different methods of assessment to the correction of a second-year exam of
...
2
votes
2answers
154 views
Reweighting features in PAM/K-means clustering
As stated in Hennig et al. 2016 Handbook of cluster analysis:
If for subject matter reasons some variables are more important than
others regardless of the within-variable variation, one could ...
8
votes
3answers
838 views
Does a statistic used as a measure becomes invalid after it's reported on? [closed]
Apologies if this the wrong place to ask. but i'm trying to explain to a director that his Statistics don't show the truth because he is demanding that the stats are what is reported on and staff are ...
0
votes
1answer
63 views
How to assess predictive validity of a variable on the outcome?
I am looking to assess the predictive validity of two measures (M1 and M2) on a binary outcome. After constructing my model, do I just calculate odds ratios and compare the odds ratios for the two ...
1
vote
0answers
17 views
Assessment of Statistical Validity
I have results from a questionnaire I have created from 32 participants. The questionnaire assesses 3 domains. I have been able to assess reliability with Cronbach's alpha but I am unsure how to ...
1
vote
0answers
36 views
Measuring emotion differentiation on a within-person level.
To measure emotion differentiation, which refers to the precision of an individual to accurately describe and distinguish between different emotional states, many studies have used a between-person ...
2
votes
1answer
84 views
Does changing the point range of items of a construct measured through likert scale affect in any way validity and reliability of the construct?
I am using a pre-existing universal tested scale to measure a specific construct. The construct is measured through 5 items on a 5 points likert scale. However, I would probably need to change the ...
0
votes
1answer
59 views
If the Pearson correlation coefficient is small/weak, can it have criterion validity?
The r value I have obtained is -0.190 and p = 0.410.
As scores on stress increase, scores on eating habits decrease.
I know from my textbook that criterion validity is about whether your measure is ...
0
votes
1answer
30 views
Validate 10% sample from population
I have two databases which include amount of passengers between two routes.
One is the full dataset, while the other is supposedly a 10% sample.
So for example, the full database will display a route ...
4
votes
1answer
74 views
Teaching examples for internal validity: published experiments later shown to be confounded in an interesting way
There are some good examples of interesting/fun spurious correlations here (Examples for teaching: Correlation does not mean causation). Many/most of these are "funny" examples that aren't likely to ...
2
votes
2answers
415 views
Why does a factor need to be internally consistent?
I'm wondering exactly what the question says: Why does a factor (as linear combination of its items) need to be internally consistent?
If I want to use a factor found in a factor analysis as a ...
0
votes
0answers
28 views
Are reliability and validity measures necessary for all parts of a survey
I am conducting analysis on a questionnaire which was a three-part study, not three separate studies. I asked the same exact set of validated scale questions for specific constructs for three ...
0
votes
1answer
31 views
Validity / reliability of instrument used by 2 people at 2 sites
How can the validity/reliability of a semi-structured interview tool be determined, conducted by two persons at two different sites separately? For instance, one person is using the tool with male ...
3
votes
1answer
92 views
Survival analysis: Excluding cases of trivial follow-up time?
I am finishing up the data collection for my retrospective cohort study, which I have designed with the intention of conducting survival analysis. Our outcomes of interest are recurrence, metastasis, ...
0
votes
0answers
57 views
Cronbach alpha application
I developed a questionnaire to measure some factors influence in decision using related literature and after some interviews. Now I have to validate and see how to distribute among the target group. I ...
0
votes
1answer
456 views
Should I calculate Cronbach's alpha for a linear regression model?
I'm interested in modelling a trait Y using variables X1, X2, X3.
I'm using a linear regression model to predict trait Y using the independent variables and therefore want to avoid multicollinearity ...
0
votes
1answer
172 views
quantitative relation between standard error of measurement and validity
Is there a function to estimate the gain in predictive validity with the decrease of the standard error of measurement?
E.g. If the short version of a test has predictive validity of 0.35 and ...
0
votes
1answer
376 views
How is independent T-Test valid for unequal samples?
I have two groups that I wish to assess for statistically significant differences (n1=1520 / n2=115). I'm actually using T-Test for independent samples.
As far as I know, there isn't a problem here ...
0
votes
2answers
294 views
How can I determine the accuracy of correlation?
When I performed correlation analysis of C02 emission with around four-hundred attributes, i found out that C02 emission and cereal production has a correlation value of 0.98 where as the most obvious ...
6
votes
3answers
302 views
Validity of using a combination of Wilcoxon tests and Spearman rho as an alternative to GLM?
I recently read a paper where the authors used either (a) a generalised linear model or (b) a combination of Wilcoxon rank sum tests and Spearman’s rank correlation, depending on whether the data ...
1
vote
0answers
13 views
Validity of conclusion of statistical tests when comparing two groups of observations in an experiment
I am just reading the results of an experiment, and I am having a hard time estimating their validity.
In the experiment, several participant had to perform a certain task twice, once with the ...
2
votes
1answer
636 views
Chi-square Discriminant Validity Test with Lavaan (R)?
I have a four factor scale that I just finished CFA on, and I was advised to use the chi-square test of differences to check for discriminant validity to reinforce my AVE based test for it. I read ...
0
votes
0answers
33 views
Does reliability/validity apply on manually summed items?
For a subscale of mine, it showed a reliability in the 0.6 range which is in the 'questionable' range and it showed an AVE (convergent validity) of just 0.3. Now I know that it will be perceived as a ...
0
votes
1answer
43 views
Convergent Validity with Multiple New Scales
I am interested in creating a new scale in a field that is overwhelmingly understudied in the research literature. I would like to test the convergent validity by modifying another survey instrument ...
0
votes
0answers
31 views
Reliability/validity required for the mean of combined items?
Let's just say that the 4 items making one construct are not well correlating (for whatever reason) and their Cronbach is 0.62 despite loading into 1 factor. Is there an alternative way to proceed? ...
1
vote
0answers
15 views
Do Standard errors and percentage of valid instances from same data set effects the F1 score?
Considering we have a data set of 944 elements. One with 45% valid instances (433 instances) and the other with 73.8% valid instances (697 instances). The F1 score ...
