An invalid questionnare with valid resultant scores?

Question

In what situation, a questionnaire could be invalid but its resultant scores could be valid? I received a comment from a journal article reviewer that says:

In keeping with the preferred terminology, the authors are reminded that the [Questionnaire] itself is not reliable and valid, but its resultant scores are. Rephrasing of this is recommended in several places in the manuscript.

What does this preferred terminology refer to? Does it refer to a general preference of attribution of reliability and validity to the resultant scores or perhaps my questionnaire poses a problem? It's good to mention that I removed 9 poor items (with low factor loadings) from 80 items of my questionnaire and the resultant scores are obtained from the remaining 71 items.

Answer 1

I think it's mostly hair splitting. The comment does not say that the tool is a bad tool. What it says is when we say something is valid or reliable, it's the measured variable (aka the score or index derived from the questionnaire) being valid or reliable, not the physical questionnaire itself. If you change the name of the questionnaire into the name of the resultant score or index (e.g. instead of "the So-and-so Questionnaire," use "So-and-so's whatever score"), then you should be fine.