2
$\begingroup$

A friend recently entered (but lost) an online competition where the winner was chosen based on total votes. Only one vote would register per IP address so as to discourage multiple votes coming from one person. The only contest rule was that "machine or automated voting" was not allowed. The following data is a small snapshot of the vote activity for the eventual winner:

Time    Total Votes <BR>
11:22   3765    <BR>
11:24   3770    <BR>
11:26   3773    <BR>
11:28   3775    <BR>
11:30   3780    <BR>
11:32   3781    <BR>
11:34   3784    <BR>
11:36   3787    <BR>
11:38   3791    <BR>
11:40   3799    <BR>
11:42   3803    <BR>
11:44   3805    <BR>
11:46   3808    <BR>
11:48   3810    <BR>
11:50   3813    <BR>
11:52   3817    <BR>
11:54   3819    <BR>
        <BR>
11:58   3827    <BR>
12:00   3829    <BR>
12:02   3834    <BR>
        <BR>
12:14   3852    <BR>
        <BR>
12:56   3990    <BR>
        <BR>
13:06   4014    <BR>
13:08   4022    <BR>
        <BR>
13:12   4029    <BR>
        <BR>
16:00   4341     Final vote count

Note that the beginning data was collected at 2 minute intervals, but later there are gaps before a large gap when the final vote count is presented.

When I put the contiguous data (11:22 - 11:54) into excel and then "fill down" for a prediction, I get a predicted final vote count = 4246... less than 100 votes from the actual count. In addition, the arrival of votes just doesn't look as random as I'd expect if independent human voting was happening. I'm pretty sure there is some degree of human voting, but I'm speculating that there is a strong machine element at play as well.

I'd sure appreciate any opinions / analysis / advice that the community can offer. Do you believe there is machine voting going on here? How strongly do you believe this?

$\endgroup$
  • $\begingroup$ What is the standard error of that prediction? $\endgroup$ – usεr11852 says Reinstate Monic Apr 30 '15 at 4:03
  • $\begingroup$ Not sure what you mean... if you mean the std deviation, the real data vs the excel "back-filled" straight-line values in the 11:22 - 11:54 range have a std of 1.66. Slope for the fitted line is about 3.4559 votes per 2 minutes. $\endgroup$ – HonestLee May 1 '15 at 1:00
  • $\begingroup$ No, I do not mean the standard deviation of your data. I mean that when you make a prediction you can get some prediction intervals about it. Anyway, I think that @BenKuhn gave you a really nice answer (+1 to him). $\endgroup$ – usεr11852 says Reinstate Monic May 1 '15 at 1:38
3
$\begingroup$

One statistical test you can do is see whether the arrival time of past votes influences the arrival time of future votes. If it does, that would strongly suggest that the votes were orchestrated in some way (e.g., a machine issuing one vote per minute, or issuing 100 votes in one minute and none for the next).

This suggests to a good null hypothesis, which is that the data are drawn from a Poisson distribution. That's our benchmark for random, independent voting patterns, where the time of one vote doesn't tell you anything about the timing of the next vote. So how do we figure out whether the data we have is Poisson-distributed?

We can try to falsify this null hypothesis--convince ourselves the data are not Poisson-distributed--by testing a statistic called index of dispersion. Basically, this measures whether the standard deviation of the data is larger or smaller than you would expect if it were a Poisson variable.

The index of dispersion is the variance (square of the standard deviation) divided by the mean. For a Poisson distribution--where voting events are independent--this ratio is always 1. If the votes are too regular and machine-like, then the variance of the vote counts, and hence the index of dispersion, will be lower. If the votes are too irregular (because they happen all at once), the index of dispersion will be higher.

We'll use just the 2-minute windows to simplify things; that means that, under the null hypothesis that the data is Poisson distributed, the number of votes in each 2-minute window comes from a Poisson distribution with the same rate parameter. We just have to find out what that rate parameter is. Our best guess is just the mean of the count in each window. I'm going to do this in Python since that's what I'm most familiar with:

In [16]: import numpy as np

In [17]: votes_per_time = [5, 3, 2, 5, 1, 3, 3, 4, 8, 4, 2, 3, 2, 3, 4, 2, ]

In [18]: np.mean(votes_per_time)
Out[18]: 3.375

So the average number of votes per window is 3.375. We can start by comparing a histogram of the plots to a true Poisson variable:

Histogram of data vs. a true Poisson variable

Hmm. It looks like it's a little bit underdispersed--concentrated towards the center, and away from the tails, compared to a Poisson distribution. But there's no smoking gun, so let's go on with the statistics. What's the index of dispersion?

In [19]: np.std(votes_per_time)**2 / np.mean(votes_per_time)
Out[19]: 0.77314814814814814

So these votes are slightly more regular (less variable) than we would expect from a Poisson distribution.

But we can't stop there. Even if the votes were really random, a sample of them wouldn't necessarily have an index of dispersion of exactly one. Instead, we need to figure out how likely it would be that, if the data were really Poisson, the index of dispersion would be as low as the one we saw. To do this, we'll create a ton of fake data that we know is from a Poisson distribution, and see how its index of dispersion goes.

In [22]: np.random.seed(1234)

In [23]: samples = [np.random.poisson(np.mean(votes_per_time), size=len(votes_per_time))
   ....:            for i in xrange(1000)]

In [24]: indices = [np.std(sample) ** 2 / np.mean(sample) for sample in samples]

Here's a density plot of that distribution:

Density of dispersion index under the null hypothesis

You can already probably see that 0.77 (the dashed line) is fairly close to the peak of this distribution; it doesn't look implausible that we'd see a value like that from truly Poisson data. We can quantify this by producing a confidence interval for the dispersion index under the null hypothesis--basically, if the data were truly Poisson, what's an interval we would be 90% sure the dispersion index would fall inside? We find this by calculating the 2.5th and 97.5th percentiles of how it would be distributed:

In [25]: np.percentile(indices, [2.5, 97.5])
Out[25]: array([ 0.38584454,  1.6878125 ])

So the index of dispersion of 0.77 that we saw above is well within the realm of possibility for truly Poisson-distributed data.

This doesn't necessarily mean that the data actually is Poisson-distributed! It might be that we just don't have a large enough sample to tell. For instance, if we saw the same index of dispersion over a sample of 1,000 2-minute windows...

In [27]: samples2 = [np.random.poisson(np.mean(evts), size=1000) for i in xrange(1000)]

In [28]: indices2 = [np.std(sample) ** 2 / np.mean(sample) for sample in samples2]

In [29]: np.percentile(indices2, [2.5, 97.5])
Out[29]: array([ 0.91544624,  1.0890494 ])

...then we could much more confidently reject the hypothesis that our data was Poisson. But since our confidence interval with the smaller number of buckets is a lot wider, we don't have as much power to detect deviations from Poisson-ness.

To sum up: if the votes are actually different from how a bunch of independent humans would make them, we don't have enough data to tell.

$\endgroup$
  • $\begingroup$ Wow @Ben ... I can't believe how awesome your answer is. Thank you for your expertise and quick reply. I think I understand most of it, but I still wonder whether it should be expected that the little bit of data collected could be extrapolated so accurately. $\endgroup$ – HonestLee Apr 30 '15 at 3:22
  • $\begingroup$ If the average rate extended over the duration, I guess the accurate prediction is expected. But I imagine with human voters, the "ebb and flow" of the day would result in the average rate changing, and thus the final total. Granted, the rate could go up and down in amounts that end up giving you the average, but that sounds less likely than a machine-driven constant average. $\endgroup$ – HonestLee Apr 30 '15 at 3:22
  • $\begingroup$ I have a request to a polling / survey company for a larger set of data, but it won't be from this particular vote. I was planning to look for this "ebb and flow", take short chunks of data, and see how accurately it predicts the outcome. If the company is kind enough to provide the data, I'll post my findings. $\endgroup$ – HonestLee Apr 30 '15 at 3:22
  • $\begingroup$ @HonestLee: if the votes are constant with rate 3.375 per 2 minutes--i.e., 1.16875 per minute--then I get that on average 278 * 1.16875 ~= 325 votes should have come in over the rest of the study, with standard deviation sqrt(325) ~= 20. So a discrepancy of 100 votes (per your calculation) or 200 (per mine) would actually be hard to explain without some degree of human "ebb and flow" to the voting rate. $\endgroup$ – Ben Kuhn Apr 30 '15 at 5:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.