# Statistical significance of a sample

Today, I was going through results of a survey when a colleague asked me, what is the statistical significance of the population cohorts in this survey.

I wanted to understand if anyone here could explain, what such a question as mentioned above would actually mean? I mean, it got me thinking and I was not able to reframe my colleague's answer in a way that would lead me to testing statistical significance.

So, let us say a population is made up of 4 segments namely - A,B,C,D. Now, if we take a sample out of this population with the 4 segments having certain sample sizes - how do we determine if these samples sizes are statistically significant. Does such a question even make sense to begin with.

What is the statistic that is of interest? What distribution do the data follow? One might want to quantify how significant. Etc.

Maybe this was clear in context (e.g. that you're trying to compare the means of some binomial distributions across four populations and your colleague trusts you to make some choices as to how significant). But without that, the question sounds a bit like "what is statistical hypothesis testing"?

To me, the question doesn't make sense because sample size is just one of the key drivers for your statistical test. To reject whatever null hypothesis you have, you may also need to consider:

• Effect size
• Design effects