In order to better understand some of the polling for the presidential election, I created a simple JS program which runs iterations of sample-based polls to prove just how accurate things like margin of error are when understanding a two person poll. While looking at the results I ran into some oddities that I can't quite explain.
Here's the scenario. If we imagine a population of 100,000,000 people that is completely split 50/50 between Trump voters and Hillary voters. It means if we were to conduct a sample poll of 1000 people, each individual would have exactly a 50/50 shot of being on one side or the other. If we desire a 95% confidence interval then using the formula of
.98 / Sqrt(sampleSize) it means we have a 3.1% margin of error. If I run 1000 iterations of this poll (sampling 1000 people with a 50/50 shot of being on one side or the other), I get numbers that confirm the confidence interval with results like in-margin/out-margin of 947/53, 961/39 etc.
Now, in this case the true ratio is 50/50. Now, lets imagine that our random sampling is a little less random than we believe, and we skew it so that our random selection method actually has a 55% chance of getting a Hillary supporter (even though the true % is 50). This means we're adding only 5% of bias off reality. Now, in order to reach a 95% confidence interval, suddenly the margin of error shifts from approximately 3.1% all the way up to ~7.5%.
So here are the questions:
Is there a formula for calculating that margin of error, given a known or guessed amount of bias off random?
Is this statistically valid at all, or am I way off on this whole escapade?
For most legit polls they try to be random. If there was known bias they would likely try and change their methods. Yet, since such a small amount of error off of random, results in a margin of error more than double the original, doesn't it call into question the accepted margin-of-error of most polls? Achieving true random feels nearly impossible, and just a little off of it, and you will be vastly missing the 95% confidence interval.
Code: https://gist.github.com/owenallenaz/6ab41a29517c3b1cad9f499c037e349f, should be able to execute in browser.