Brexit: was "leave" statistically significant? In this post we ask a question about a natural phenomenon called humans attempt to find decision by counting votes. The specific incident of such natural phenomenon that this question is about is the case of Brexit. 
Note: the question is not about politics. The goal is to try to discuss such natural phenomenon from a statistical point of view based on observations.
The specific question is:


*

*Question: What does the $51.9\%$ Brexit vote to leave mean? E.g. does it mean that the public really wants to leave EU? Does it simply mean that the public is unsure and needs more time to think? Or is it something else?


Assumption 1: there is no error in the voting process.
 A: 51.9% is the percentage of voters who want to leave.  Since the sample size is so large (>33 million), there is virtually no random sampling error.
Statistical significance testing would try to determine if the difference in remain and leave could be explained by random sampling error alone, and the difference would certainly be significant (see @caveman's answer).
The problem with this approach is that statistical significance makes a very strong assumption that the sample is representative of the entire population (all of Britain), not just those who vote.
The non-response rate (those that do not vote) is enormously important in determining if more than half of all of Britain wants to 'leave', and is difficult to measure.  Non-response bias is created when subgroups who are less likely to vote have systematically different views.  Based on exit-polls, for example, millennials were less likely to vote, but more likely to vote to remain, which biases the results when trying to represent the population of all of Britain.
For this reason, statistical significance testing in its traditional sense is largely inappropriate.

Assumptions:  We need to define some terms for any of this to make sense and avoid political discussion of what voting is trying to accomplish.  Here are my definitions:
Population:  Every person living in Britain
Sampling Frame:  Every voting eligible person capable of voting
Sampling Methodology:  Voluntary response, the act of voting is participating in the survey
Sample:  The individuals who actually vote
In this setup, the sample proportion could be used (for better or worse) to estimate the percentage of all people who lean towards remain (or leave).
A: You ask

What does the 51.9% Brexit vote to leave mean? 

It means 51.9% of the voters voted to leave.  

E.g. does it mean that the public really wants to leave EU? Does it simply mean that the public is unsure and needs more time to think? Or is it something else?

The votes comprised $17\,421\,887$ "leave" votes and $16\,146\,297$ "remain" votes, indicating $12\,931\,353$ eligible voters did not vote and approximately $18$ million inhabitants are not eligible voters.  Since neither the collection of actual voters nor the collection of eligible voters is "the public" and neither is a representative (random, unbiased, pick a relevant adjective) sample of "the public", the 51.9% Brexit vote is noninforming to your second and subsequent questions.
It might have been possible to construct a questionnaire responsive to your questions.  This does not seem to have been what happened in the referendum as implemented.
A: You could ask a slightly different question: Assuming that 50% of a very large population voted "Yes", and you asked a random sample of size S, what is the probability that 51.9% of your sample responded "Yes", depending on the sample size? 
The expected value of number of "Yes" votes is 0.5 S. The variance is 0.25 S. The standard definition is 0.5 $S^{1/2}$. A deviation of the actual from the expected number of "Yes" votes more than 6.1 standard deviations has a chance of one in a billion. 
We have this when 0.019 S (difference between 50% and 51.9%) is 6.1 * 0.5 * $S^{1/2}$, or S = $(6.1 * 0.5 / 0.019)^2$ or S ≈ 25,800.  
A: I agree with @Underminer that there is no sampling error, but not because sample is large, but because there was no sampling involved. Nobody was sampled to vote. There obviously was some negligible fraction of people who wanted to vote but, weren't able to (e.g. had car accident on this day), or who made invalid votes, but that's the only "sampling" in here.
The result is exact, there is no error involved since the whole population took part in vote (some took part by not taking part in it). Some people decided to vote, some didn't. Some decided to vote on leave, some didn't. Democracy is not about statistical significance, but about what really happened. Voting is not intended to learn about people opinion, but to make a decision. Actually, people sometimes do not vote according to what they think, but to manifest, or achieve something. For example, in election people may vote not to their preferred candidate, but to their second preferred one if they think he has greater chances of winning.
A: This is another solution using an analytical method instead of a simulation.
Previously, I have simulated an unsure population to be one that its vote is random chance guessing. So out of $n$ many voters, an unsure population would tend to vote leave or remain for $0.5$ of the time.
In order for an unsure population to get exactly $51.9\%$ vote on leave, there needs to be $17,421,887$ 1s in $O_{\text{leave}}$. The probability for this is $0.5^{33,568,184}$. Similarly, the probability of getting $17,421,887 + 1$ votes is also $0.5^{33,568,184}$. This goes on.
This is the probability of getting $\ge 17,421,887$ votes:
$$\begin{split}
\sum_{i=17,421,887}^{33,568,184} 0.5^{33,568,184} &= (33,568,184-17,421,887) \times 0.5^{33,568,184}\\
&= 8.39663381928984×10^-10105024\\
&\approx 0\\
\end{split}
$$
($8.39663381928984×10^-10105024$ calculated by Wolframalpha)
And this is the probability of having $\ge 51.9\%$ of an unsure population vote leave.
