How to choose and perform a goodness-of-fit test? I notice there are a lot of questions on the topic (most are along the lines of: how to do a Chi-squared goodness-of-fit test), but mine is perhaps more general.
Using an example I recently encountered. I have a one-sample dataset, where the data are strictly positive, and have a right-skewed distribution. I make a guess that the data may be fitted to one of the four distributions:


*

*Log-normal

*Exponential

*Weibull

*Gamma


At first, I assess the distribution visually using geom_histotgram (from package ggplot2) and plot the estimated distributions using stat_function, where the distribution parameters are estimated using fitdistr (from package MASS).
ggplot(DT4, aes(TTE)) + geom_histogram(aes(y=..density..), bins=90) +
  stat_function(fun=dlnorm,args=fitdistr(DT4[, TTE],"log-normal")$estimate, color=pal[1], size=.5) +
  stat_function(fun=dexp,args=fitdistr(DT4[, TTE],"exponential")$estimate, color=pal[2], size=.5) +
  stat_function(fun=dweibull,args=fitdistr(DT4[, TTE],"weibull")$estimate, color=pal[3], size=.5) +
  stat_function(fun=dgamma,args=fitdistr(DT4[, TTE],"gamma")$estimate, color=pal[4], size=.5) +
  annotate("text", label="Log-normal", x=250, y=0.025, color=pal[1]) +
  annotate("text", label="Exponential", x=250, y=0.023, color=pal[2]) +
  annotate("text", label="Weibull", x=250, y=0.021, color=pal[3]) +
  annotate("text", label="Gamma", x=250, y=0.019, color=pal[4])


All four distributions have a faily good fit, with log-normal perhaps having the best fit, just from a visual assessment.
However, imagine you are a professional statistician (which I am not) who wants to assess the fit rigorously. I have looked around and found numerous goodness-of-fit tests, including


*

*Chi-squared / G-test (designed for discrete data, but may be applied to continuous data if it is divided into arbitrary bins)

*Kolmogorov-Smirnov

*Cramér–von Mises

*Anderson-Darling

*Maximum Likelihood Estimate

*Information Criteria: AIC/AIC$_\text{c}$/CAIC/BIC/HQIC

*Any others I have missed?


Question


*

*How do we choose which goodness-of-fit to use to compare the fit of different distributions? In straightforward cases, one distribution may have the best fit according to every test across the board, but in more marginal cases, what if one distribution is best according to one test, and another is best according to another test? (Is there a reference, like a book chapter, that explores this question?)

*Is there a go-to R function/package that conveniently performs a range of goodness-of-fit tests all at once?
 A: Great question! (Almost a professional statistician here!) Let's see if I can try to answer your question. Whuber gave a comment on what the typical strategy that someone might follow. I'll try to complement that by providing a couple papers that talk specifically about properties of tests when assumptions are violated. I'm doing this since, if I had a choice between multiple goodness of fit tests, I would do a process of elimination by saying which tests are less "valid" given my data and potential sources of noise. 
When it comes to goodness of fit tests, there a few different criteria to keep in mind, and assumptions on the estimated distributions tend to matter - the degree to which they matter (i.e., robustness to assumption violations varies). Some of the papers below talk about validity and goodness of fit under different assumption violations.
Estimating Parameters in Kolmogorov Smirnov Test
On the Kolmogorov Smirnoff test, for example, in the empirical-to-theoretical distribution comparison, if you have to estimate parameters to conduct the test, then the p-values are technically not valid. The paper below gives a quick review on this. 
Lilliefors, H. W. (1969). On the Kolmogorov-Smirnov test for the exponential distribution with mean unknown. Journal of the American Statistical Association, 64(325), 387-389.
https://amstat.tandfonline.com/doi/pdf/10.1080/01621459.1969.10500983?needAccess=true#.Xh3UTFNKhQI
Chi-Square Test with Heterogeneous variance
Usually in regression, we assume constant variance. Sometimes, this isn't the case, but there are robust alternatives to the standard chi-square. This paper talks about a statistic that is asympototically chi-square but is robust to non-constant variance as well. 
Lipsitz, Stuart R., and John F. Buoncristiani. "A robust goodness‐of‐fit test statistic with application to ordinal regression models." Statistics in medicine 13, no. 2 (1994): 143-152.
https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.4780130205
Using Mahalonobis Distance to mitigate against Outliers
Cerioli, Andrea, Alessio Farcomeni, and Marco Riani. "Robust distances for outlier-free goodness-of-fit testing." Computational Statistics & Data Analysis 65 (2013): 29-45.
https://www.sciencedirect.com/science/article/pii/S016794731200134X
Goodness of Fit R package
Here are a couple of packages that perform specific goodness of fit tests. I don't know if there exists one that does them all at once? I will keep looking and see if I can find one. 


*

*GOFT:
https://www.tandfonline.com/doi/abs/10.1080/00949655.2017.1404604?src=recsys&journalCode=gscs20

*GOFTEST:
https://cran.r-project.org/web/packages/goftest/goftest.pdf
