A test (for example, psychological) should be designed in a manner to get valid information, i.e. it must be valid instrument. Is there any relevance of reliability in constructing a test, too? What is the difference between reliability and validity? They both seem to be often based on correlations, so it is easy to mix the concepts.
$\begingroup$
$\endgroup$
2
asked Jan 3, 2014 at 7:41
-
$\begingroup$ This question was re-worded by me to some extent to make it more consistent. I believe the question deserves some upvoting to above "zero level" of votes because it happened to get some upvoted answers already. $\endgroup$– ttnphnsCommented Sep 19, 2017 at 10:14
-
$\begingroup$ @ttnphns, done. I will delete this comment later. Probably all the comments related to that discussion should be deleted. $\endgroup$– mpiktasCommented Sep 25, 2017 at 7:48
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
9
Reliability has to with being able to replicate results. So if you apply the same measurment technique multiple times in similar situations you should get similar results. An unreliable measurement adds a lot of random noise to your measurement.
Validity has to do with measuring what you want to measure. This is a much more theoretical concept: if this is a survey you just need to think about what the questions are, how a respondent might interpret that question (and the possible answer categories), the theoretical concept you want to measure, and whether these all match up.
-
$\begingroup$ validity is not a theoretical concept simply. It can be assessed and checked ie cross-validated $\endgroup$– user10619Commented Jan 7, 2014 at 11:16
-
$\begingroup$ If I wanted to know the anual cheese consumption in the Netherlands and I measured the number of ants per square meter in Berlin, then that would not be a very valid measure of annual cheese consumption. How would cross-validation help us to detect that? $\endgroup$ Commented Jan 7, 2014 at 13:11
-
$\begingroup$ You can always deceive a naïve person with statistics. A statistician hopefully understands the meaning of "target population" and thus takes a sample from the similar or equivalent population. $\endgroup$– user10619Commented Jan 10, 2014 at 17:44
-
3$\begingroup$ This is not about the difference between statisticians and the rest nor is this just about sampling from the right population. Validity is about measuring what you want to measure, and getting the sample wrong is just one way in which you can fail to achieve that goal. More common is to just ask the wrong question or a question that is misunderstood by the respondents. Looking at your data (with more or less advanced statistical techniques) is unlikely to definitively show whether or not you have a problem. Most often you will just need to sit down and think about it to determine it. $\endgroup$ Commented Jan 11, 2014 at 13:54
-
$\begingroup$ I believe one should examine the same or equivalent population even if we have different settings. Cross validation does not imply that we should not account for given structure. $\endgroup$– user10619Commented Jan 11, 2014 at 16:55
$\begingroup$
$\endgroup$
5
Validity of a psychometric scale is its correlation with the trait supposed being measured (called criterion), and uncorrelation with extraneous systematic factors.
Reliability of a psychometric scale is its correlation with its own "true" score, and uncorrelation with extraneous random factors.
This is what @Maarten said, put more formally.
Often seen as two different facets of an instrument functioning, reliability and validity sometimes are difficult to separate conceptually from each other. As seen for example with the term "internal consistency"$^1$. They seem to be two different (and perhaps competitive) paradigms about correlations observed among the items constituting the scale. Reliability stance believes that the measured trait is simple, unifactor structure; each item measures it and just it + is liable to some random error. The lesser is the error the more item-item homogeneous is the scale. Validity stance regards the overall score by the scale as if it were the external criterion; validity is interested in that item-to-total correlation, rather than in how well items duplicate each other. Thus, what the reliability approach will tend to pre-include into the error term, the validity approach will ascribe to the complexity, the multi-aspect, multi-symptom nature of the trait (and decide whether it is still the same construct or is already a battery of different ones). If "systematic error" were easy to distinguish from "random error" without assumptions, there would be no frictions between reliability and validity.
$^1$ Cronbach's alpha is one of measures of internal consistency from reliability standpoint.
-
2$\begingroup$ Thanks Sir!,I am agree with your answer.Basically reliability is authenticity of a construct /Test/Questionnaire while validity is the degree of trust worthiness ,soundness and accuracy of data that gives information out of it( test/questionnaire). $\endgroup$ Commented Jan 4, 2014 at 8:07
-
$\begingroup$ I guess that this is a problem with English, but I would say it the other way around. $\endgroup$ Commented Jan 4, 2014 at 9:44
-
$\begingroup$ @ttnphns Though your answer is clear to a person who has an idea of psychometry and statistics. It is comprehensive. can CFA (confirmatory factor analysis) help us compute validity coefficient. $\endgroup$– user10619Commented Jan 7, 2014 at 10:55
-
$\begingroup$ Generally, yes. It depends on how you use CFA and what is a the criterion for you. $\endgroup$– ttnphnsCommented Jan 7, 2014 at 11:40
-
$\begingroup$ thanks for your valuable comments $\endgroup$ Commented Aug 29, 2023 at 15:50