Although this is my own question, I'm also going to post my own two-cents as an answer, so that we add to the number of perspectives on this question. The issue here is whether or not it is sensible to initially fit a one-parameter distribution to data. When you use a one-parameter distribution (such as the Poisson GLM, or a binomial GLM with fixed trial parameter), the variance is not a free parameter, and is instead constrained to be some function of the mean. This means that it is ill-advised to fit a one-parameter distribution to data in any situation where you are not absolutely sure that the variance follows the structure of that distribution.
Fitting one-parameter distributions to data is almost always a bad idea: Data is often messier than proposed models indicate, and even when there are theoretical reasons to believe that a particular one-parameter model may obtain, it is often the case that the data actually come from a mixture of that one-parameter distribution, with a range of parameter values. This is often equivalent to a broader model, such as a two-parameter distribution that allows greater freedom for the variance. As discussed below, this is true for the Poisson GLM in the case of count data.
As stated in the question, in most applications of statistics, it is standard practice to use distributional forms that at least allow the first two moments to vary freely. This ensures that the fitted model allows the data to dictate the inferred mean and variance, rather than having these artificially constrained by the model. Having this second parameter only loses one degree-of-freedom in the model, which is a tiny loss compared to the benefit of allowing the variance to be estimated from the data. One can of course extend this reasoning and add a third parameter to allow fitting of skewness, a fourth to allow fitting of kurtosis, etc. The reason that these higher-order moments are not usually as important is that asymptotic theorems for estimators usually show that they converge to a normal distribution (regardless of the higher-order moments of the underlying data) and in this case estimates of the mean and variance are sufficient to get good estimates of the asymptotic distribution of the parameter estimators.
With some extremely minor exceptions, a Poisson GLM is a bad model: In my experience, fitting a Poisson distribution to count data is almost always a bad idea. For count data it is extremely common for the variance in the data to be 'over-dispersed' relative to the Poisson distribution. Even in situations where theory points to a Poisson distribution, often the best model is a mixture of Poisson distributions, where the variance becomes a free parameter. Indeed, in the case of count data the negative-binomial distribution is a Poisson mixture with a gamma distribution for the rate parameter, so even when there are theoretical reasons to think that the counts arrive according to the process of a Poisson distribution, it is often the case that there is 'over-dispersion' and the negative-binomial distribution fits much better.
The practice of fitting a Poisson GLM to count data and then doing a statistical test to check for 'over-dispersion' is an anachronism, and it is hardly ever a good practice. In other forms of statistical analysis, we do not start with a two-parameter distribution, arbitrarily choose a variance restriction, and then test for this restriction to try to eliminate a parameter from the distribution. By doing things this way, we actually create an awkward hybrid procedure, consisting of an initial hypothesis test used for model selection, and then the actual model (either Poisson, or a broader distribution). It has been shown in many contexts that this kind of practice of creating hybrid models from an initial model selection test leads to bad overall models.
An analogous situation, where a similar hybrid method has been used, is in T-tests of mean difference. It used to be the case that statistics courses would recommend first using Levene's test (or even just some much crappier "rules of thumb") to check for equality of variances between two populations, and then if the data "passed" this test you would use the Student T-test that assumes equal variance, and if the data "failed" the test then you would instead use Welch's T-test. This is actually a really bad procedure (see e.g., here and here). It is much better just to use the latter test, which makes no assumption on the variance, rather than creating an awkward compound test that jams together a preliminary hypothesis test and then uses this to choose the model.
For count data, you will generally get good initial results by fitting a two-parameter model such as a negative-binomial or quasi-Poisson model. (Note that the latter is not a real distribution, but it still gives a reasonable two-parameter model.) If any further generalisation is needed at all, it is usually the addition of zero-inflation, where there are an excessive number of zeroes in the data. Restricting to a Poisson GLM is an artificial and senseless model choice, and this is not made much better by testing for over-dispersion.
Okay, now here are the minor exceptions: The only real exceptions to the above are two situations:
(1) You have extremely strong a priori theoretical reasons for believing that the assumptions for the one parameter distribution are satisfied, and part of the analysis is to test this theoretical model against the data; or
(2) For some other (strange) reason, the purpose of your analysis is to conduct a hypothesis test on the variance of the data, and so you actually want to restrict this variance to this hypothesised restriction, and then test this hypothesis.
These situations are very rare. They tend to arise only when there is strong a priori theoretical knowledge about the data-generating mechanism, and the purpose of the analysis is to test this underlying theory. This may be the case in an extremely limited range of applications where data is generated under tightly controlled conditions (e.g., in physics).
