Measuring Difference when AB Testing is not ideal Normally it is best to be able to set up a randomized AB test to measure if some change is actually better than the original.  What about in situations where it is not ideal to AB test?  
For example, we are a ride sharing company operating in an area where the amount of drivers are much lower than the amount of people looking for rides, and we want to test a new pricing algorithm and measure the impact of it.  Splitting our users into a control and test group will make it more difficult to meet the already higher than supply demand, and different pricing between drivers may incite negative feedback and complaints on the uneven payments.  There can also be many difficulties when it comes to controlling variables such as distances from drivers to riders while trying to randomize our test and control groups.  
In events like this, which technique would be ideal to be able to measure a difference with some degree of certainty?
 A: Edited to reflect revised interpretation of question
If the algorithms were selected based on some mathematical model of consumer behavior, that same model might be used to evaluate the new algorithm and compare its projections directly to the old algorithm. If no model currently exists, one could be developed and, if it performs well enough predicting usage under the current algorithm, then used to estimate effects of the new algorithm.
Without such a model, I might do something like conduct a survey of potential customers asking about how much they might use the ride share service under the current surge pricing algorithm as well as the new one. Their responses regarding the current algorithm can be compared against data collected from real observations of use (also under the current algorithm) to estimate how well the stated preferences match actual use. The answers to questions about the new algorithm allow for estimation of the new algorithm's performance, mediated by the differences between stated and revealed preferences regarding the current algorithm.
I don't doubt that there are a lot of other approaches, but this strikes me as a question that's hard to answer in the general case. Modeling is a formal way of examining the relevant factors' effects on the outcome, but your predictions will only be as accurate as the underlying model. Surveys are effective in some applications and less so in others. I don't know if there is a single "ideal" technique for projecting the effect of a potential change like you describe unless the question is substantially narrowed.
