I faced a challenge recently with our commercial team about our annual market survey result. As you can guess, our commercial team has no background in statistic or math etc.
A brief introduction, after careful sampling design based on population and stratified probability sample from national phone database bank, we selected randomly phone numbers and interviewed 10,000 people. The stratas are specified by Geography (18 regions) and within stratas the phone numbers were selected randomly within strata quota requirements were implemented by interlacing Age (10-19; 20-29; 30-39; 40-49; 50-59; 60-69, 69+) and Gender (Males, Females). We asked people if they listen/stream podcasts or stream/watch film on Netflix/HBO and if yes how often (daily, weekly, monthly etc). I weighted the survey based on official population data and produced a report.
One of our competitors run a similar survey with 12,000 samples (as far as I am aware) with a similar method via phone. Our result does not match and it came as no surprise to me. For example, we are saying 10% of persons aged 10 and older listening to a podcast or watch video on Netflix in a typical month but our competitor's report says 14% listen or watch daily. or our result shows 59% of the population watch/stream at least once a month, but their survey says 46% etc.
And this became such a headache for me, our sales team keeps questioning why we report lower or higher compared to company X survey. I tried to explain them two different surveys from two different companies will not give the exact same result, As we are sampling different samples and we should expect a change on the results as this is claim based. also explain there is a sampling error in each survey. For example if the reality of gender distribution is 50%, when I sampled and ask people about their age, there is a X% sampling error. well, It didn't work since I guess they cannot follow my explanation.
So I was wondering if anyone can help me to better explain or formulate the why question for folk who has no idea how survey and sampling work