Why is Poisson regression used for count data? I understand that for certain datasets such as voting it performs better. Why is Poisson regression used over ordinary linear regression or logistic regression? What is the mathematical motivation for it? 
 A: Poisson distributed data is intrinsically integer-valued, which makes sense for count data. Ordinary Least Squares (OLS, which you call "linear regression") assumes that true values are normally distributed around the expected value and can take any real value, positive or negative, integer or fractional, whatever. Finally, logistic regression only works for data that is 0-1-valued (TRUE-FALSE-valued), like "has a disease" versus "doesn't have the disease". Thus, the Poisson distribution makes the most sense for count data.
That said, a normal distribution is often a rather good approximation to a Poisson one for data with a mean above 30 or so. And in a regression framework, where you have predictors influencing the count, an OLS with its normal distribution may be easier to fit and would actually be more general, since the Poisson distribution and regression assume that the mean and the variance are equal, while OLS can deal with unequal means and variances - for a count data model with different means and variances, one could use a negative binomial distribution, for instance.
A: Mathematically if you start with the simple assumption that the probability of an event occurring in a defined interval $T = 1$ is $\lambda$ you can show the expected number of events in the interval $T = t$ is is $\lambda.t$, the variance is also $\lambda.t$ and the probability distribution  is
$$p(N=n) =  \frac{(\lambda.t)^{n}e^{-\lambda.t}}{n!}$$ 
Via this and the maximum likelihood method & generalised linear models (or some other method) you arrive at Poisson regression.
In simple terms Poisson Regression is the model that fits the assumptions of the underlying random process generating a small number of events at a rate (i.e. number per unit time) determined by other variables in the model.
A: Others have basically said the same thing I'm going to but I thought I'd add my take on it.  It depends on what you're doing exactly but a lot of times we like to conceptualize the problem/data at hand.  This is a slightly different approach compared to just building a model that predicts pretty well.  If we are trying to conceptualize what's going on it makes sense to model count data using a non-negative distribution that only puts mass at integer values.  We also have many results that essentially boil down to saying that under certain conditions count data really is distributed as a poisson.  So if our goal is to conceptualize the problem it really makes sense to use a poisson as the response variable.  Others have pointed out other reasons why it's a good idea but if you're really trying to conceptualize the problem and really understand how data that you see could be generated then using a poisson regression makes a lot of sense in some situations.  
A: Essentially, it's because linear and logistic regression make the wrong kinds of assumptions about what count outcomes look like.  Imagine your model as a very stupid robot that will relentlessly follow your orders, no matter how nonsensical those orders are; it completely lacks the ability to evaluate what you tell it.  If you tell your robot that something like votes is distributed continuously from negative infinity to infinity, that's what it believes votes are like, and it might give you nonsensical predictions (Ross Perot will receive -10.469 votes in the upcoming election).
Conversely, the Poisson distribution is discrete and positive (or zero... zero counts as positive, yes?).  At a very minimum, this will force your robot to give you answers that could actually happen in real life.  They may or may not be good answers, but they will at least be drawn from the possible set of "number of votes cast".
Of course, the Poisson has its own problems: it assumes that the mean of the vote count variable will also be the same as its variance.  I don't know if I've ever actually seen a non-contrived example where this was true.  Fortunately, bright people have come up with other distributions that are also positive and discrete, but that add parameters to allow the variance to, er, vary (e.g., negative binomial regression).
A: My understanding is primarily because counts are always positive and discrete, the Poisson can summarize such data with one parameter. The main catch being that the variance equals the mean.
