The best answer I've come across can be found in To Explain or To Predict? (Shmueli 2010). In the article, the author breaks down the choices that an analyst makes (the "two modeling paths") depending on whether their goal is prediction or explanation (i.e., inference). This is not a complete list of considerations when building models, but it does provide a basis for understanding how a predictive model and an explanatory model may differ given the same data.
In short, explanatory models focus on minimizing bias (i.e., the difference between the expected and true value of an estimator) in order to more accurately represent the underlying relationship between X and Y, whereas predictive models minimize both bias and estimation variance (std. error). Additionally, explanatory problems require that the model's coefficients are interpretable (for somewhat obvious reseasons), whereas predictive problems often sacrifice interpretability for greater predictive power (e.g., using 'black box' algorithmic and nonparametric models).
More to my initial question: Explanatory models are evaluated using 'goodness of fit' tests (e.g., R-squared, Mallow's Cp, etc.) and other model diagnostics (e.g., residual analysis) that "[measure] the strength of the relationship indicated by f-hat" (p 16). Evaluating predictive models involves comparing the performance of the model on the training vs the holdout datasets. The predictive model should minimize the error on the test set (or else the model may be over-fitted). Additionally, explanatory models may have to worry about identifying sources of endogeneity, collinearity, etc., which increase bias in the model. These concerns are minimized for predictive models.
Finally, theoretical considerations (e.g., expected causal relationships) play a role in model evaluation, though more so for explanatory models. For example, "a researcher might choose to retain a causal covariate which has a strong theoretical justification even if is statistically insignificant" (p 17). This would never be done in a predictive model because it would decrease its predictive power. Similarly, because interpribility is paramount in explanatory models, these may include additional variables that do nothing for predictive power (e.g., main and interaction terms).
All in all, though it is obviously important to know what sort of problem you have before starting your analysis, explanation models and prediction models are two sides of the same coin. There is always a tension between minimizing variance and bias and analysts need to consider what techniques minimize the stat/s that will create the best model.
Note: f refers to the model function, where F refers to an underlying function that describes the true relationship between X and Y. In other words, the true relationship between X and Y is:
Y = F(X),
whereas the statistical model is:
E(Y) = f(X),
where X and Y are operationalizations of X and Y.