Answer is book length. Topic is too broad. For example, some of the items listed feature tests that are not comparable to others on the list, and in some the similarities are narrow between broad topics. It would be easier to ask "How do I process data," which although also very broad as a question, has the advantage of being more organized.
Sic...
Step one for examining data; Cartesian plot it.
Step two; find a general organizing principle for the data, that simplifies it, e.g., but not limited to, linearizing the data or finding homoscedasticity. This includes, but is not limited to, data transforms like log-log plotting, semi-log plotting, linear-log plotting, polar coordinate plotting, reciprocation, squaring or square-rooting.
Step three; fit models to the transformed data using the the criteria of interest for modelling. Sometimes the criteria of interest involve goodness of fit, sometimes goodness of fit is irrelevant. For example, if we wish to stabilize scaled data with respect to fitting a semi-infinite support distribution like a scaled gamma distribution, we may wish to use ridge regression by optimizing parameters that cause the total area to have a least relative error. That would have little in common with goodness of fit and would not be directly related to an AIC (Akaike information criterion), which in turn would limit the considerations to maximum likelihood fitting, i.e., goodness of fit. Another example, we may wish to have a best predictive model, e.g., one that least error predicts out of range withheld data. That would also be different from a goodness of fit criterion within the training set.
Step four; compare multiple models under the criteria of interest for usefulness.
Why this lecture? When we execute step three above, we must apply our purpose or reason for modelling. That purpose will imply criteria for model selection and allow us to the select a battery of tests that comparatively or absolutely characterize the performance of a model or models.
If you want a more specific answer, you must select a purpose for your intended modelling. How the listed criteria fit into the steps above depends on when they are applied. For example, following step one above, if we find that $r^2<<r_s^2$ (Pearson-squared much less than Spearman-rank squared), we may want to linearize the data by data transformation, and if we find that $r^2>>r_s^2$, the data may have undesirable outlier properties, or generally a relatively sparse middle x-axis population resulting in false augmentation of the apparent correlation. After data transformation, even if the data appears to be linear in Cartesian coordinates, the $r^2$ value of the transformed data may increase. This could occur, for example, when the data is proportional error type and we take the logarithm of the y-axis values. However, correlation is hardly the only criterion one would apply to make a decision regarding the data. Ideally, correlation goes to precision, but not accuracy, of modelling. A/B testing and ANOVA are similar and are subject to the same general considerations as correlations. For example, adjusted R$^2$, partial ANOVA probabilities, AIC, AIC$_c$ and BIC, principal component analysis, weighted least squares, and others are directed towards model selection, and which is(are) useful depends on the goal, target, reason, motive, designated usage or what have you in step four above.
