I have a questionnaire with ~40 questions where the respondent answers on a scale 1-7 on each question. At the moment we have a sample size of say 10,000 people. The question is how I calculate the amount of data we lose by lowering the sample size and asking 2,000 people (representative from the whole sample) instead? How much would that change the answers? Can we expect the average of a question to (for example) go from 5.2 to $5.2 \pm 0.3$?
We know from experience that the answers are not normally distributed and are more skewed to the higher values if that helps. Say that a question might have average 5.2 and standard deviation of 0.4.
Or the other way around: For example, if we had gotten an average on a question of 5.2 if we had ask 10,000 people, and we say that we can accept a deviation +/- 0.1 if we lower the sample size, would that mean that we could ask 2000 instead, or 5000 or 6000?
It all comes down to the usual fact that it will be cheaper to ask fewer people.
