Should predictive accuracy or, alternatively, minimizing the MSE, be reconsidered? Ever since Breiman, maximizing predictive accuracy has become a predictive modeling gold standard, of sorts. That it has evolved to this status is understandable:  it can be "optimized," is easily calibrated across k-fold samples and, for the most part, is a consistent statistic for internal model validation. However, all too often it is the sole criterion for model value -- a mechanistic and rigidly applied metric -- despite its being prone to p-hacking, gaming, and analyst fraud as well as little or no thought being given to its wider business and/or strategic impact and meaning.
Is it time to re-evaluate this metric's relative importance in the statistical, machine learning and predictive modeling industries?
 A: This is a nice and thoughtful post, and in my working life I have observed the things you outline to be correct - the successful statisticians and scientists at my workplace are those that can step back from raw predictive accuracy and deliver a model or analysis that is holistically appropriate for the problem at hand.  Sometimes this is raw predictive power, but often it is not.
I explicitly look for this when interviewing, my favorite initial answer to a modeling question is

Well, it depends...

I'll add some examples to your list.
Implementation Costs 
Many businesses run core systems on outdated technology, cobol or fortran codebases running on ancient mainframe architectures.  They are often reluctant to replace them because of the high fixed costs for doing so (even though the variable costs of maintaining them are high).  This can have drastic consequences for model implementation.  It may be possible to get a predictive boost from a random forest or gradient booster, but implementing a model of that complexity in a production environment can be completely infeasible.
Shelf Life 
Related to implementation costs, a model, once implemented, may have a very long shelf life, and be expected to deliver reasonable predictions for a long time.  A model with maximum supportable complexity fit very hard to the data is less robust to distribution shifts in the population and predictive relativity changes between segments.
Tinkering 
Business people have a tendency to tinker with production models, and we as modelers sometimes have to assist with hot fixes in production systems.  Complex models are more sensitive to this, it is harder to accurately asses how they will react to a production adjustment (talk to a mechanic about whether its easier to get under the hood of a car manufactured in 1980 vs. 2010).
Robustness to New Information 
A categorical predictor may obtain new categories in the future, and is often desirable to have principled way to deal with these without refitting a model and pushing it to production.
Model Componentization 
A model may be part of a larger system optimization, which imposes environmental constraints on its form and properties.  One common source of this is when a model is a component of a larger mathematical optimization scheme, with some causal predictor in the model being manipulated as a lever to enhance business results.  This can impose smoothness or differentiability constraints on the predictors that are very important to consider.
Locality Constraints 
Some models have better locality properties than others.  For example, if I wish to estimate the price elasticity of a customer for small adjustments, then a highly local model (i.e. a density smoother with small bandwidth, a regularized spline with small parameter, or a gradient tree booster with lots of cuts) will invariably use less of the data to support inferences even on a local scale.  This can be undesirable when these inferences are used to make important decisions, and should be supported by as much data as possible.
