Andrew Gelman likes to write about this topic and has been posting extensively about it lately on his blog. I don't always agree with him but I like his perspective on $p$-hacking. Here is an excerpt from the Introduction to his Garden of Forking Paths paper (Gelman & Loken 2013; a version appeared in American Scientist 2014; see also Gelman's brief comment on the ASA's statement), emphasis mine:
This problem is sometimes called “p-hacking” or “researcher degrees of freedom” (Simmons, Nelson,
and Simonsohn, 2011). In a recent article, we spoke of “fishing expeditions [...]”. But we are starting to feel that the term “fishing” was unfortunate, in that it invokes an image
of a researcher trying out comparison after comparison, throwing the line into the lake repeatedly
until a fish is snagged. We have no reason to think that researchers regularly do that. We think
the real story is that researchers can perform a reasonable analysis given their assumptions and
their data, but had the data turned out differently, they could have done other analyses that were
just as reasonable in those circumstances.
We regret the spread of the terms “fishing” and “p-hacking” (and even “researcher degrees of
freedom”) for two reasons: first, because when such terms are used to describe a study, there is
the misleading implication that researchers were consciously trying out many different analyses
on a single data set; and, second, because it can lead researchers who know they did not try
out many different analyses to mistakenly think they are not so strongly subject to problems of
researcher degrees of freedom. [...]
Our key point here is that it is possible to have multiple potential comparisons, in the sense of
a data analysis whose details are highly contingent on data, without the researcher performing any
conscious procedure of fishing or examining multiple p-values.
So: Gelman does not like the term p-hacking because it sort of implies that the researches were actively cheating. Whereas the problems can occur simply because the researchers choose what test to perform/report after looking at the data, i.e. after doing some exploratory analysis.
With some experience of working in biology, I can safely say that everybody does that. Everybody (myself included) collects some data with only vague a priori hypotheses, does extensive exploratory analysis, runs various significance tests, collects some more data, runs and re-runs the tests, and finally reports some $p$-values in the final manuscript. All of this is happening without actively cheating, doing dumb xkcd-jelly-beans-style cherry-picking, or consciously hacking anything.
So to the extent that "p-hacking" is understood broadly a la Gelman's forking paths, the answer to how prevalent it is, is that it is almost universal.
The only exceptions that come to my mind might be pre-registered replications in psychology and perhaps medical trials.
$P$-value distributions in the literature
I have not heard about Head et al. study before, but have now spent some time looking through the surrounding literature. I have also taken a brief look at their raw data.
Head et al. downloaded all Open Access papers from PubMed and extracted all p-values reported in the text, getting 2.7 mln p-values. Out of these, 1.1 mln was reported as $p=a$ and not as $p<a$. Out of these, Head et al. randomly took one p-value per paper but this does not seem to change the distribution, so here is how the distribution of all 1.1 mln values looks like (between $0$ and $0.06$):
I used $0.0001$ bin width, and one can clearly see a lot of predictable rounding in the reported $p$-values. Now, Head et al. do the following: they compare the number of $p$-values in the $(0.045, 0.5)$ interval and in the $(0.04, 0.045)$ interval; the former number turns out to be (significantly) larger and they take it as an evidence of $p$-hacking. If one squints, one can see it on my figure.
I find this hugely unconvincing for one simple reason. Who wants to report their findings with $p=0.05$? Actually, many people seem to be doing exactly that, but still it appears natural to try to avoid this unsatisfactory border-line value and rather to report another significant digit, e.g. $p=0.048$ (unless of course it's $p=0.052$). So some excess of $p$-values close but not equal to $0.05$ can be explained by researcher's rounding preferences.
And apart from that, the effect is tiny.
(The only strong effect that I can see on this figure is a pronounced drop of the $p$-value density right after $0.05$. This is clearly due to the publication bias.)
Unless I missed something, Head et al. do not even discuss this potential alternative explanation. They do not present any histogram of the $p$-values either.
There is a bunch of papers criticizing Head et al. In this unpublished manuscript Hartgerink argues that Head et al. should have included $p=0.04$ and $p=0.05$ in their comparison (and if they had, they would not have found their effect). I am not sure about that; it does not sound very convincing. In this preprint on PeerJ the same Hartgerink et al. extract p-values from lots of papers in top psychology journals and do not find reliable evidence of $p$-hacking. They do some weird blunders such as choosing 0.00125 bin width in their Figure 2 to make a histogram that has regular peaks with 0.001 spacing (I suspect it creates artifact peaks). But they also do some nice stuff, like recomputing exact $p$-values from the reported $t$-, $F$-, $\chi^2$- etc. statistic values; this distribution cannot have any rounding artifacts and does not exhibit any increase towards 0.05 whatsoever (Figure 4):
Based on all that, my conclusion is that I don't see any good evidence of $p$-hacking in $p$-value distributions across biological/psychological literature as a whole.
It seems that the first to observe an alleged excess of $p$-values just below 0.05 were Masicampo & Lalande 2012:
This does look impressive, but Lakens 2015 (preprint) in a published Comment argues that this only appears impressive thanks to the misleading exponential fit. See also Lakens 2015, On the challenges of drawing conclusions from p-values just below 0.05 and references therein.
Even though the evidence presented by Head et al. and in the related studies is not particularly convincing, there is hardly any doubt that "benign" $p$-hacking is extremely widespread. As I wrote above, my own anecdotal observations certainly support it. Amusingly, some people even polled researchers to find that many admit doing some sort of hacking (John et al. 2012, Measuring the Prevalence of Questionable
Research Practices With Incentives for
And also, everybody heard about the so called "replication crisis" in psychology: more than one half of the recent studies published in the top psychology journals do not replicate (Nosek et al. 2015, Estimating the reproducibility of psychological science). (This study has recently been all over the blogs again, because the last issue of Science published a Comment attempting to refute Nosek et al. and also a reply by Nosek et al. The discussion continued elsewhere, see today's post by Andrew Gelman and the RetractionWatch post that he links to. To put it politely, the critique is unconvincing.)
So I would say that we know that there must be a lot of $p$-hacking going on, mostly of the Forking-Paths type that Gelman describes; probably to the extent that published $p$-values cannot really be taken at face value and should be "discounted" by the reader by some substantial fraction. However, this attitude seems to produce much more subtle effects than simply a bump in the $p$-values distribution just below $0.05$ and cannot really be detected by such a blunt analysis.