I conduct pre/post analyses looking at a health metrics (such as BMI) for a sample population. The sample might be around 75 people for a total population of about 500. One of the questions I get all the time is how much does this change represent the overall population (who have not been part of the analysis).

I have been thinking about ways to do this, looking at MOE of the sample and if the error bars overlap between pre and post, then there is no statistically detectable difference at the aggregate population level. Is this the best way to do this?

  • $\begingroup$ Please tell us what data you are using. You say "... for populations," suggesting you are using a complete census, but then the question about representativeness indicates you only have a sample. Which is it? What kind of sample? What basis do you have for supposing that internal sample-based calculations like error bars should tell you anything about the population itself? What do you mean by "aggregate population level"? $\endgroup$ – whuber Nov 27 '17 at 21:07
  • $\begingroup$ thanks for the feedback. i made some changes-- i am looking at a sample that represents a larger population. its my understanding the MOE measures the margin by which the sample metric could be off from the true value in overall population within some number of std devs. if so, comparing the error bars in both the pre and post would be indicative of whether there is a detectable difference in the true population value. am i right in thinking this way? $\endgroup$ – eljusticiero67 Nov 27 '17 at 21:27
  • $\begingroup$ Unfortunately, that logic is circular. The MOE has an interpretation akin to the one you give only when the sample is random (which is representative). In a non-representative sample by definition there's no way to relate properties of the sample to properties of the population. That's why you cannot answer your question solely by examining the data: you must have additional information about how the data might be related to the population in the first place. $\endgroup$ – whuber Nov 27 '17 at 22:27
  • $\begingroup$ Let's say it is random. What would that tell us then? $\endgroup$ – eljusticiero67 Nov 27 '17 at 22:29
  • $\begingroup$ Then you're done: you have a theoretical basis to make inferences from properties of the sample to properties of the population using any applicable statistical procedure. $\endgroup$ – whuber Nov 27 '17 at 22:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.