An instrument is valid if it measures what it purports to measure. This is different from reliability, which means that an instrument measures (whatever it measures) in a way that is repeatable. DO NOT use this tag for `validation` model purposes.

learn more… | top users | synonyms

0
votes
0answers
18 views

Is there a minimal percentage of a data set to have the property to be modelled?

We have a data set of about 45,000 observations. Around 2 % of these have the property that we're interested in modelling. We have made a probit model fitted to all 45,000 observations and get ...
1
vote
0answers
4 views

How to obtain statistical information from a Likert scale with a sample size smaller than the number of questions?

Research and sample properties I'm measuring the degree of a certain concept related to marketing and sustainable development throughout an entire industry. The theoretical model is depicted next: ...
0
votes
0answers
17 views

Reliability & Validity of modified /adapted scale

I have modified a scale applicable for employees in organizations to athletes in sports. Except changing a few words applicable for the target population no other changes are brought. The four ...
0
votes
0answers
25 views

Is my survey invalid now?

I'm running an online survey for my thesis. In order for the survey to be valid, I needed a minimum of 375 participants. And I have achieved this number. However, upon closer inspection I have ...
2
votes
1answer
25 views

General validity/reliability of questionnaires

I have a general question concerning the reliability/validity of questionnaires. I'm working in Human Resources and we plan to conduct a workplace engagement survey. I attended a conference and a ...
0
votes
0answers
35 views

Principal Component Analysis: Scale reliability and validity

I'm using a popular psychometric scale, the Domain Specific Risk Taking (DoSpeRT) scale, that contains 30 items that measure an individual's risk taking capacity across five different dimensions: ...
0
votes
0answers
12 views

Access validity and reliability for hypotheses testing in prediction model

In my bachelor thesis I'm investigating a transaction data set of a retailer. I'm trying to find out indicators that predict a certain outcome, which is why I proposed several hypotheses for the ...
2
votes
1answer
89 views

Alternative methods for logistic regression

Usually the condition of the validity of a logistic regression is to have 10 events per predictor. In our model the binary outcome variable (1 if Healthy aging ; 0 otherwise) has a frequency of ...
0
votes
0answers
13 views

What sort of cross validation is this?

I've always tested my classification techniques using non-standardised trial and error but I'm interested to see which category my techniques fall under. They seem to fall under several but I'm not ...
0
votes
0answers
22 views

How to tell if the survey is good enough/valid?

I just used 15-items questionnaires X which has never been used in my culture C. I've found that in my sample (n=150) α = .77, principal component analysis with 5 factors more than 1 eigenvalue, ...
0
votes
0answers
9 views

Concurrent validity between two tests

I need to set up a study to gather concurrent validity for a new assessment tool, which is intended to assess fine motor skill performance. I will use a "gold-standard" test as a way of comparison. I ...
1
vote
0answers
14 views

How can I compare the stability of a variable with another variable over time?

I have an instrument that measures two independent samples daily. If I know that the measurements of the first sample are accurate in a time period what is the best method to validate the measurements ...
1
vote
0answers
37 views

How can measure be unreliable but valid?

Some previous research found evidence for the predictive validity of a measure. I've done some work and found the measure to be unreliable. Since reliability provides an upper limit for predictive ...
3
votes
2answers
75 views

What tradeoffs are there between internal and external validity?

I was reading through an article about research design. The authors gave an example on the relation between internal and external validity which I thought might be an important area of consideration ...
0
votes
1answer
208 views

How to justify the choice of independent variables in multiple regression

I am trying to measure the effect of atmospheric factors as smell or light (IV) on purchase behavior (DV). In total I have xx likert scales that contain 5 likert items and responses are coded from 1 ...
1
vote
0answers
44 views

How to justify construct validity when items dropped from established instrument?

I conducted a survey in which I used the Major Depression Inventory (MDI). While analyzing this instrument (as my Depression construct) in SmartPLS, I iteratively dropped items that had non-...
0
votes
0answers
42 views

Reliablity and validity new questionnaire

A questionnaire with three subtopics concerning; well being, competence and empathy is adjusted and translated by us. We would like to know whether our new, shorter version of this questionnaire is ...
1
vote
1answer
42 views

What is “content validity”?

I'm a university student studying statistics by 'Discovering statistics using ibm spss 4th edition' I don't understand what 'content-validity' means even though I read the explanation from the book ...
0
votes
0answers
21 views

Diagnostic regression for checking the validity of clustering results

I have done an unsupervised non-parametric clustering on sample data gathered by a questionnaire for my thesis (k-means algorithm). A referee asked me to do a diagnostic regression for checking the ...
0
votes
0answers
22 views

Is there literature on removing questions from validated questionnaire for research?

I wish to use a validated questionnaire for research purposes but two (of the 26) questions would not be relevant in my country. The would make it a 24 item questionnaire. It is to assess knowledge ...
0
votes
0answers
67 views

Validity of pre-established scale when dropping item as suggested by CFA

During my CFA I was forced to eliminate one item from the market orientation scale (well-established scale) due to poor loading which reduces the model fit. My PhD examiners asked me to explain the ...
4
votes
0answers
59 views

What's an intuitive way to explain the different types of validity?

Specifically I'm thinking of a simplified division whereby validity is divided into: Construct validity 1a. Convergent validity 1b. Discriminant validity Criterion related validity 2a. Predictive ...
4
votes
2answers
1k views

Cronbach's Alpha in Pilot Study: do I need to eliminate items if alpha is already > 0.7

In the Pilot Study, I am trying to confirm reliability of my scales: I'm performing Cronbach’s alpha test on 5 items that represent one construct of my research model (IV). I'm receiving α = .710 ...
2
votes
1answer
72 views

How to test the inter-rater reliability of a nominal coding process that generates ordinal data?

A researcher studying hunger on a tropical island asks respondents, "What did you eat for lunch yesterday?" and the possible choices are "(4) Nothing; (3) a banana; (2) an unripe coconut; (1) a ripe ...
1
vote
1answer
43 views

Where I can find resources to learn how to calculate the sample size representativeness, and realiability and validity of questionnaires?

I'm totally newbie in psychometrics but starting from a research paper with full data I would like to understand: how to calculate if the sample size is representative; how to calculate reliability ...
1
vote
1answer
34 views

An invalid questionnare with valid resultant scores?

In what situation, a questionnaire could be invalid but its resultant scores could be valid? I received a comment from a journal article reviewer that says: In keeping with the preferred ...
0
votes
0answers
34 views

Is a representative sample necessary to assess construct validity?

To generalize results to a greater population, a representative sample is necessary. In developing a psychometrically valid instrument, via confirmatory factor analysis or item response theory, we are ...
6
votes
1answer
544 views

How high is too high with Cronbach's alpha?

Someone in my organization just sent me a scale they developed (modified items from five separate scales used in published research) to measure an employee's readiness for change. They tested the ...
4
votes
1answer
32 views

Is this method of arriving at a statistic valid?

The following is an excerpt from an article about gender bias : Overall, 34 per cent of respondents readily admitted that their family preferred having more boys. Given the sensitive nature of ...
0
votes
0answers
38 views

Validity in IRT Style

From my reading, I found that IRT generally provides more sophisticated information. Therefore, in regards of unidimensional 3PL IRT model, how should I report the finding about validity in ...
0
votes
0answers
56 views

How can you judge the statistical confidence and validity of output from a multi arm bandit algorithm like UCB1

To say something about the validity of outcomes in frequentist statistics we have concepts like significance levels and statistical power and in Bayesian analytics we have credible intervals. In a ...
5
votes
2answers
44 views

Validity of combining surveys and sociodemographics to estimate miles walked in census tracts

I ran across this paper that describes an interesting approach to combining survey and sociodemographic data: Salon. Estimating Total Miles Walked and Biked by Census Tract in California. State of ...
0
votes
1answer
177 views

Is 0.40 Alpha score (KR-20) acceptable for a true-false test?

I have 10 true-false-Idont know test items. I codded correct responses as 1, incorrect ones and dont knows as 0. I got 0.40 alpha score (or KR-20). Do you think that this score is acceptable for such ...
1
vote
2answers
377 views

EFA or CFA for validity

I need to test validity of measurement scales in my survey. 1) Is it true that both can be used to test measurement validity? 2) Or there are some types of validity that can be tested with EFA and ...
1
vote
0answers
211 views

How to measure construct validity?

I recently received feedback from a journal (in education) that "The idea that you can assess construct validity through a factor analysis is inconsistent with how we usually think about validity in ...
1
vote
0answers
29 views

Factor analysis without theoretical congruence

I've been working with certain personality theory, which I'm trying to measure in a sample of students. It's a foreign theory, so I created an instrument for my specific population. I got a Cronbach's ...
1
vote
0answers
20 views

Choosing between two scales

I am investigating a scale of consumer ethnocentrism - CETSCALE. I do not have much knowledge in econometrics, scales, etc., however: a) research papers report that the standard 17-item scale is ...
2
votes
1answer
71 views

households survey data - determine 'valid' product categories

I'm working with household survey data where households report their expenditure for various products. The goal is to look into income elasticities for various products thus in essence I'm interested ...
0
votes
0answers
618 views

Covariance between factors in CFA (Amos)

I ran a Confirmatory Factor Analysis (CFA) in Amos and obtained a model with high covariance between the factors. James described this covariance in his video as alarming. What can I do to tackle ...
3
votes
1answer
279 views

Evaluation measures of overlapping clustering

I have a dataset of Facebook users and a set of different clustering algorithms. The project goal is to draw up a rank between these algorithms in order to understand which of them are the good ones. ...
1
vote
0answers
112 views

Validity of a statistical procedure to calculate an odds ratio for a Fisher's exact test?

I am trying to determine whether this is a valid statistical procedure. I have ~1000 samples, and each sample has around 1-500 data points. Each data point consists of two fractions (value1 and ...
0
votes
0answers
55 views

Selection Bias with Groups

I have conducted a pre-post test study with a set of 20 participants. There were two independent variables and therefore 4 groups of 5 participants each. The experiment took place in a classroom and ...
11
votes
2answers
404 views

Identification of useless questions from a questionnaire

I'm developing a questionnaire. To improve its reliability and validity I want to use statistical methods. I want to eliminate questions whose answers are always the same. This means that nearly all ...
4
votes
4answers
2k views

If a measure is valid (but not necesarily reliable), can it be consistently replicated?

I had this question on an exam, and I was positive that the answer was A. The Beck Depression Inventory is a scale intended to measure depression levels, with higher scores indicative of higher ...
2
votes
0answers
54 views

Small data set and validity of Multilevel Regression results

I want to estimate a hierarchical multilevel regression (two levels). My data set is fairly small: 25 groups and 255 individual observations distributed unequally among groups (group member size ...
0
votes
2answers
131 views

Do reliability and construct validity have same technical(statistical) assumptions?

It appears that reliability and construct validity have a similar backing in terms of statistics. Is it true ?
2
votes
1answer
704 views

CFA Fit Measure Thresholds for New Scales

A well established questionnaire measuring interpersonal relations is modified to measure interorganizational relations. 11 parameters of the relation are determined by 4 items each. Items are removed ...
0
votes
1answer
212 views

Can we use CFA (confirmatory factor analysis) to ascertain validity coefficient?

Validity coefficient is an important measure for test development especially in psychology and sociology wherein we have too indirect measurements. Perhaps we have different ways to know its size.
-1
votes
2answers
5k views

What is the difference between reliability and validity?

A test should be designed in a manner to get valid information. Is there any relevance of reliability in constructing a test? What is the difference between reliability and validity?
3
votes
2answers
636 views

How does reliability and validity affect the results (descriptive statistics)?

Okay, I think this might be a stupid question but I just can't quite seem to get there. I understand what reliability and validity is, but I don't understand just what I should do with it when looking ...