Demonstration of DOE of mixture design applied to 2-stage gain schedule for machine learner tl;dr
Please provide an example of statistical design of experiment (DOE) of "mixture design" of learning rates to 2-stage gain schedules for machine learners.
Background
Statistical DOE is a rigorous approach that helps to get the most information from experiments performed.  It gives a rigorous structure to the approach that makes it high yield.
According to NIST, factors of a "Mixture Design" are described as:

In a mixture experiment, the independent factors are proportions of
different components of a blend.

In machine learning, for many optimizers, there is a "learning rate" value that increases quality either by reducing time to learn or reducing distance from estimated "best" to actual "best" at the end of the process. One popular optimizer is "Adam", and its variable is called "learning rate", but its adaptation is dynamic.  If possible I would like to use piecewise constant learning rates (like this).
Also in machine learning there is a "rule of thumb" that when learning rate is modulated, in general it is better to go from higher values to lower values.  (See learning rate adaptation here and here)
If the number of epochs is fixed (100 epochs), then training using a sequence of of gains sorted from largest to smallest can be considered a variation on a "mixture design".  If half of all epochs are at learning rate 0.9, and the other at 0.1, then the mixture proportions are 90% and 10%, respectively.
Question:
Are there any existing examples of an approach like this being used to design rate schedules in machine learners?
For demo case:
An acceptable example could be using MNIST and a MLP (like this) except with gain schedules DOE applied to the learning rates.
 A: So I've let this sit to aggregate comments or answers.  Nobody is doing that so I will.
Note: this is a work in progress.  I leave this note until I am done.
Thoughts:

*

*There is a need to repeat runs so the randomization-based variation
can show itself.  Perhaps 10 replications at every level should give
a sense of center and spread for the settings.

*A useful approach is to exploit adaptive methods like those for
MatLab's "fplot" to perform search along a single axis.  It could
take much more compute but it should give a "ground truth" to compare
the DOE approach and results against.

Preparation

*

*baseline accuracy in learning (classify MNIST using NN, run 10x, look at convergence, ...)

*Make naive split in momentum and run it once as proof of code convergence, accuracy, epochs.

*use fplot to determine a general form of the function (step 5 in design)

Problem statement:

*

*define the goal: find the reliably reached convergence-level accuracy (from baseline) in the fewest epochs

*select the components: mixture of fixed learning rates sorted in descending order, where mixture portion is percent of total epochs at learning rate.  For each replication the initial weights on the network are randomly initialized (Hu?)

*identify constraints: total epochs is fixed, learner is fixed, inputs are fixed, software is RStudio/R, keras, my computer, ...

*Response variable of interest is 90th percentile of epochs to convergence accuracy across replicates.  Smaller is better.

*Reasonable: Exponential decay to a floor of the form $ f ~ a + b*exp(-c*i) $ where a, b, and c are constants and i is iteration number, should be a reasonable fit, but it strongly begs the question.

*use DOE on the above for test of model adequacy and to find best parameters

New work goes here <---
