The utility of effect sizes relative to p-values (as well as other metrics of statistical inference) is routinely debated in my field—psychology—and the debate is currently “hotter”, than normal for reasons that are relevant to your question. And though I am sure psychology isn’t necessarily the most statistically sophisticated scientific field, it has readily discussed, studied—and at times, demonstrated—limitations of various approaches to statistical inference, or at least how they are limited by human use. The answers already posted include good insights, but in case you are interested in a more extensive list (and references) of reasons for and against each, see below.
Why are p-values undesirable?
- As Darren James notes (and his simulation shows), p-values are largely contingent on the number of observations that you have (see Kirk, 2003)
- As Jon notes, p-values represent the conditional probability of observing data as extreme or more extreme given that the null hypothesis is true. As most researchers would rather have probabilities of the research hypothesis, and/or the null-hypothesis, p-values do not speak to probabilities in which researchers are most interested (i.e., of the null or research hypothesis, see Dienes, 2008)
- Many who use p-values do not understand what they mean/do not mean (Schmidt & Hunter, 1997). Michael Lew’s reference to Gelman and Stern’s (2006) paper further underscores researcher misunderstandings about what one can (or cannot) interpret from p-values. And as a relatively recent story on FiveThirtyEight demonstrates, this continues to be the case.
- p-values are not great at predicting subsequent p-values (Cumming, 2008)
- p-values are often misreported (more often inflating significance), and misreporting is linked to an unwillingness to share data (Bakker & Wicherts, 2011; Nuijten et al., 2016; Wicherts et al., 2011)
- p-values can be (and historically, have been) actively distorted through analytic flexibility, and are therefore untrustworthy (John et al., 2012; Simmons et al., 2011)
- p-values are disproportionately significant, as academic systems appear to reward scientists for statistical significance over scientific accuracy (Fanelli, 2010; Nosek et al., 2012; Rosenthal, 1979)
Why are effect sizes desirable?
Note that I am interpreting your question as referring specifically to standardized effect sizes, as you say they allow researchers to transform their findings “INTO A COMMON metric”.
- As Jon and Darren James indicate, effect sizes indicate the magnitude of an effect, independent of the number of observations (American Psychological Association 2010; Cumming, 2014) as opposed to making dichotomous decisions of whether an effect is there or not there.
- Effect sizes are valuable because they make meta-analyses possible, and meta-analysis drive cumulative knowledge (Borenstein et al., 2009; Chan & Arvey, 2012)
- Effect sizes help to facilitate sample size planning via a priori power analysis, and therefore efficient resource allocation in research (Cohen, 1992)
Why are p-values desirable?
Though they are less frequently espoused, p-values have a number of perks. Some are well-known and longstanding, whereas others are relatively new.
P-values provide a convenient and familiar index of the strength of evidence against the statistical model null hypothesis.
When calculated correctly, p-values provide a means of making dichotomous decisions (which are sometimes necessary), and p-values help keep long-run false-positive error rates at an acceptable level (Dienes, 2008; Sakaluk, 2016) [It is not strictly correct to say that P-values are required for dichotomous decisions. They are indeed widely used that way, but Neyman & Pearson used 'critical regions' in the test statistic space for that purpose. See this question and its answers]
- p-values can be used to facilitate continuously efficient sample size planning (not just one-time power-analysis) (Lakens, 2014)
- p-values can be used to facilitate meta-analysis and evaluate evidential value (Simonsohn et al., 2014a; Simonsohn et al., 2014b). See this blogpost for an accessible discussion of how distributions of p-values can be used in this fashion, as well as this CV post for a related discussion.
- p-values can be used forensically to determine whether questionable research practices may have been used, and how replicable results might be (Schimmack, 2014; also see Schönbrodt’s app, 2015)
Why are effect sizes undesirable (or overrated)?
Perhaps the most counter-intuitive position to many; why would reporting standardized effect sizes be undesirable, or at the very least, overrated?
- In some cases, standardized effect sizes aren’t all that they are cracked up to be (e.g., Greenland, Schlesselman, & Criqui, 1986). Baguely (2009), in particular, has a nice description of some of the reasons why raw/unstandardized effect sizes may be more desirable.
- Despite their utility for a priori power analysis, effect sizes are not actually used reliably to facilitate efficient sample-size planning (Maxwell, 2004)
- Even when effect sizes are used in sample size planning, because they are inflated via publication bias (Rosenthal, 1979) published effect sizes are of questionable utility for reliable sample-size planning (Simonsohn, 2013)
- Effect size estimates can be—and have been—systemically miscalculated in statistical software (Levine & Hullet, 2002)
- Effect sizes are mistakenly extracted (and probably misreported) which undermines the credibility of meta-analyses (Gøtzsche et al., 2007)
- Lastly, correcting for publication bias in effect sizes remains ineffective (see Carter et al., 2017), which, if you believe publication bias exists, renders meta-analyses less impactful.
Echoing the point made by Michael Lew, p-values and effect sizes are but two pieces of statistical evidence; there are others worth considering too. But like p-values and effect sizes, other metrics of evidential value have shared and unique problems too. Researchers commonly misapply and misinterpret confidence intervals (e.g., Hoekstra et al., 2014; Morey et al., 2016), for example, and the outcome of Bayesian analyses can distorted by researchers, just like when using p-values (e.g., Simonsohn, 2014).
All metrics of evidence have won and all must have prizes.
American Psychological Association. (2010). Publication manual of the American Psychological Association (6th edition). Washington, DC: American Psychological Association.
Baguley, T. (2009). Standardized or simple effect size: What should be reported?. British Journal of Psychology, 100(3), 603-617.
Bakker, M., & Wicherts, J. M. (2011). The (mis) reporting of statistical results in psychology journals. Behavior research methods, 43(3), 666-678.
Borenstein, M., Hedges, L. V., Higgins, J., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex, UK: John Wiley & Sons, Ltd.
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2017, August 12). Correcting for bias in psychology: A comparison of meta-analytic methods. Retrieved from osf.io/preprints/psyarxiv/9h3nu
Chan, M. E., & Arvey, R. D. (2012). Meta-analysis and the development of knowledge. Perspectives on Psychological Science, 7(1), 79-92.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159.
Cumming, G. (2008). Replication and p intervals: p values pre- dict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science, 3, 286– 300.
Dienes, D. (2008). Understanding psychology as a science: An introduction to scientific and statistical inference. New York, NY: Palgrave MacMillan.
Fanelli, D. (2010). “Positive” results increase down the hierarchy of the sciences. PloS one, 5(4), e10068.
Gelman, A., & Stern, H. (2006). The difference between “significant” and “not significant” is not itself statistically significant. The American Statistician, 60(4), 328-331.
Gøtzsche, P. C., Hróbjartsson, A., Marić, K., & Tendal, B. (2007). Data extraction errors in meta-analyses that use standardized mean differences. JAMA, 298(4), 430-437.
Greenland, S., Schlesselman, J. J., & Criqui, M. H. (1986). The fallacy of employing standardized regression coefficients and correlations as measures of effect. American Journal of Epidemiology, 123(2), 203-208.
Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E. J. (2014). Robust misinterpretation of confidence intervals. Psychonomic bulletin & review, 21(5), 1157-1164.
John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. PsychologicalSscience, 23(5), 524-532.
Kirk, R. E. (2003). The importance of effect magnitude. In S. F. Davis (Ed.), Handbook of research methods in experimental psychology (pp. 83–105). Malden, MA: Blackwell.
Lakens, D. (2014). Performing high‐powered studies efficiently with sequential analyses. European Journal of Social Psychology, 44(7), 701-710.
Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28(4), 612-625.
Maxwell, S. E. (2004). The persistence of underpowered studies in psychological research: causes, consequences, and remedies. Psychological methods, 9(2), 147.
Morey, R. D., Hoekstra, R., Rouder, J. N., Lee, M. D., & Wagenmakers, E. J. (2016). The fallacy of placing confidence in confidence intervals. Psychonomic bulletin & review, 23(1), 103-123.
Nosek, B. A., Spies, J. R., & Motyl, M. (2012). Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7(6), 615-631.
Nuijten, M. B., Hartgerink, C. H., van Assen, M. A., Epskamp, S., & Wicherts, J. M. (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior research methods, 48(4), 1205-1226.
Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638-641.
Sakaluk, J. K. (2016). Exploring small, confirming big: An alternative system to the new statistics for advancing cumulative and replicable psychological research. Journal of Experimental Social Psychology, 66, 47-54.
Schimmack, U. (2014). Quantifying Statistical Research Integrity: The Replicability-Index. Retrieved from http://www.r-index.org
Schmidt, F. L., & Hunter, J. E. (1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In L. L. Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if there were no significance tests? (pp. 37–64). Mahwah, NJ: Erlbaum.
Schönbrodt, F. D. (2015). p-checker: One-for-all p-value analyzer. Retrieved from http://shinyapps.org/apps/p-checker/.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological science, 22(11), 1359-1366.
Simonsohn, U. (2013). The folly of powering replications based on observed effect size. Retreived from http://datacolada.org/4
Simonsohn, U. (2014). Posterior-hacking. Retrieved from http://datacolada.org/13.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143(2), 534-547.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve and effect size: Correcting for publication bias using only significant results. Perspectives on Psychological Science, 9(6), 666-681.
Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PloS one, 6(11), e26828.