It appears that reliability and construct validity have a similar backing in terms of statistics. Is it true ?
$\begingroup$
$\endgroup$
2
-
$\begingroup$ Your title asks one question, your text asks a different question. $\endgroup$– Peter FlomCommented Jan 13, 2014 at 12:28
-
$\begingroup$ what is the minimum sample required for a construct validity through PCA (using a PLS)? is 40 enough for example? $\endgroup$– user48362Commented Jun 17, 2014 at 11:31
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
No, reliability and construct validity do not have a similar purpose.
Reliability is about whether a test measures something accurately and reliably. Validity is about whether it measures what it purports to measure.
answered Jan 13, 2014 at 12:27
-
$\begingroup$ I think validity here reflects a different thinking and still converges into reliability. It could be equivalent groups from different schools that generate approximately equal results say in terms of I.Q. or weight. If you want to check for validity (here meaning cross-validation, you need to have dissimilar groups from the same school or from other school with dissimilar characteristics. $\endgroup$– user10619Commented Jan 13, 2014 at 12:48
-
1$\begingroup$ No, it does not. Reliability can be assessed based on a single measure on one sample on one test (e.g. split half reliability) or two measures on the same sample on the same test (test retest reliability). Validity cannot. $\endgroup$ Commented Jan 13, 2014 at 13:26
$\begingroup$
$\endgroup$
8
To expand a bit on @PeterFlom : Think of a target at which an individual can shoot.
Reliability means that the shots are all close together, but it does not necessarily mean that they have to be close to the bullseye.
Validity means that the shots are on average close to the bullseye, but it does not necessarily mean that the shots are close together.
So a valid measure can be reliable or not and a reliable measure can be valid or not; all combinations are perfectly feasible. So these are really two separate characteristics of a measurement.
-
2$\begingroup$ My PhD is in psychometrics and @Maarten is correct. $\endgroup$ Commented Jan 13, 2014 at 13:24
-
1$\begingroup$ @Maarten, Albeit I concur with your response, on the overall, your metaphor about valididty is questionnable. Do you want to say that sending shots to 1 m left and 1 m right from the bullseye is as "valid" as sending them to 1 cm left and 1 cm right from it? $\endgroup$– ttnphnsCommented Jan 13, 2014 at 14:05
-
1$\begingroup$ Yes, that is exactly what I mean. Your example shows that it is a good idea to look at more than one criterium. In this example it would help to also look at reliability. Based on that criterium you would obviously prefer the ones that are cms apart. It becomes an interesting tradeoff when the cms appart are somewhat less valid than the ones ms apart. $\endgroup$ Commented Jan 13, 2014 at 19:55
-
1$\begingroup$ @Maarten, your sense of validity as, actually, unbiasedness, is uncommon and counterintuitive. It appears that very blunt instrument is as valid as a sharp one, if they are both unbiased. In fact, the correct definition of valid series of shots is the series which is as close as possible to the bullseye and as far as possible from some "false bullseye" which you want not to hit. $\endgroup$– ttnphnsCommented Jan 14, 2014 at 8:59
-
1$\begingroup$ P.S. The problem with your explanation in the example herein is that you treat deviations from the bullseye (the misses) as random errors. But validity is defined vis-a-vis systematic errors only (such as "false" bullseyes around the true one, which possibly are the projections of the shooter's idiosyncratic technical flaws on the target). It is reliability, not validity way of thinking, which deals with random errors. $\endgroup$– ttnphnsCommented Jan 14, 2014 at 9:23