Offline evaluation of Counter factual data for Recommendation I am building new model and facing at offline evaluation tasks. 
My goal is to predict higher CTR(=click/impression) advertisement, and improve sales.(sales would improve if user watch more attractive ads and click more than before)
At this moment, I have log data which is biased by current running rule; high estimated CTR ads appeared in the logs more often than the low estimated CTR ads.
user_id | ad_id | click_flg
    111 | A     | 1
    111 | B     | 0
    111 | C     | 1
    111 | B     | 0
    222 | A     | 1
    222 | C     | 0

How can I evaluate new model to current running rule in terms of sales without A/B test???
As far as I search, It seems that I can apply counter factual frame works. 
Also, I am wondering if I can use replay method of bandit algorithm with counter factual. 
I am still reading paper, but Risk Minimization for this tasks exists. 
I would appreciate for any help such as comment, link. Cheers. 
 A: You can use inverse propensity scoring (IPS) to evaluate new models if you have access to the probability each particular ad was shown. The idea is to weight the reward by something that controls for the bias in your ad-display process. Let's say your data looks like
$$(x_1, y_1, \delta_1), \ldots, (x_n, y_n, \delta_n)$$
where $x_i$ is the context (user information), $y_i$ is the action (the ad shown) and $\delta_i$ is the reward (the binary click value). Denote the model that you gathered the data under by $\pi_0$, so that $\pi_0(y|x)$ is the probability you show ad $y$ given context $x$. 
Now, to test a new model $\pi$, the IPS estimator is given by
$$\hat{U}_{ips}(\pi) = \frac1n\sum_{i=1}^n \delta_i \frac{\pi(y_i|x_i)}{\pi_0(y_i|x_i)}$$
This will give you an unbiased estimate of expected reward for your new model based on data gathered under the old model. In other words, this is the reward you expect to have gotten if you had deployed $\pi$ instead of $\pi_0$. 
The reason why this works is that the weight adjusts for the sampling bias in $\pi_0$. For example, suppose an observation is unlikely in the old model and likely in the new model, then the weight $\frac{\pi(y_i|x_i)}{\pi_0(y_i|x_i)}$ will be large. This makes sense because we probably only have a few reward samples for this observation under the old model, while we would expect to see it more frequently in the new model, so we should weight it higher. The same logic works in the reverse, relatively less frequent observations in the new model will result in a lower weight. 
I would suggest this SIGIR 2016 tutorial by Thorsten Joachims for more information on counterfactual evalution. 
