Estimating the mean with previous knowledge I have an unknown (discrete) probability distribution $p=\{p_s\}$, where $p_s$ is the probability of observing configuration $s$. To each configuration is associated an energy that I can compute $E_s$.
If I want to estimate the mean 
$$
\langle E \rangle_p := \sum_s E_s p_s \ \ \ \  (1)
$$
by drawing $N$ samples, I will clearly use the sample mean estimator:
$$
M_E := \frac{1}{N} \sum_j E_j  \ \ \ \ (2)
$$
where $E_j$ is the energy that I obtained at the $j-$th draw. 
At this point something happens and my distribution $p_s$ is changed just a bit, but in an unknown way, i.e., the new distribution is
$$
p_s \to p'_s = p_s +\delta p_s.
$$
The energy of each configuration is unchanged. 
Now I would like to estimate the new average
$$
\langle E \rangle_{p'} := \sum_s E_s p'_s
$$
Is there a way to do that, that takes into account that I have some knowledge of $p'$ [namely, I already estimated (1) via (2)]? The goal is to minimize the number of samples $N$ that need to be taken. 
EDIT
Let me add something as to why this may not be hopeless.
On the one hand, once my distribution changed, without taking any additional sample I can simply guess the average with the previous estimation. The question is essentially if I can do better than that?
On the other hand I can assume that my perturbation is of order $\epsilon$, can I obtain an estimation of the new mean up to the same order (at least approximately)?
I'd be interested in any reference or even a no-go theorem or no-go argument.
EDIT 2
I was hoping that something like Kalman filtering or Bayesian inference could do the trick but I know too little in that field.
 A: As others have pointed out in the comments, the specification that a variable is "changed in an unknown way" does not give any clear information to that can be used in your present problem.  Moreover, the goal of your problem is unclear.  If your goal is just to estimate the expectation arising from the second probability vector then there is no reason to take any samples from the first at all - just wait until the change and then sample directly from the new probability vector to estimate its corresponding expected energy.  On the other hand, if there is some reason to estimate both of the expected energy values, then you will need to formulate clearly what your overall goal is.  If you want to find an optimal sampling mechanism then you will need to formulate this as a clear optimisation problem.
In view of all this, rather than trying to extract information from your question that isn't there, I am going to instead show you how you might go about formulating this as a well-specified optimisation problem representing the kind of situation you are describing.  The formulation I will give is just an example, and can be varied in all sorts of ways.  Nevertheless, it might give you a baseline to start to think about how you want to formulate your problem so that you have a well-specified optimisation.  In particular, the formulation I give will ensure that the choice of the number of sample values from the second probability vector (after the change) is affected by the inference from the first probability vector, which seems to be what you are asking about.

Bayesian Dirichlet model: Consider a hierarchical Bayesian Dirichlet model specified by:
$$\mathbf{p}'|\mathbf{p} \sim \text{Di}(\kappa \cdot\boldsymbol{p}) \quad \quad \quad \quad \quad
\mathbf{p} \sim \text{Di}(\boldsymbol{\alpha}),$$
where the values $\kappa$ and $\boldsymbol{\alpha}$ are hyper-parameters.  Suppose you have the option of sampling from $\mathbf{p})$ prior to the change in probability, and then sampling from $\mathbf{p}'$ after this change.  Specifically, suppose you make a one-time decision to sample $n$ values from the former, and then after observing the values from this sample you can make a one-time decision to sample $n'$ values from the latter.  The observed vectors are:
$$\mathbf{X} \sim \text{Multinomial}(n,\mathbf{p}) \quad \quad \quad \quad \quad 
\mathbf{X}' \sim \text{Multinomial}(n',\mathbf{p}').$$
Your goal is to estimate the expected energy for each of the probability vectors, with some overall loss function that depends on your estimates and the true values.  To make the problem non-trivial we impose a cost on the observations, and this also factors into the loss function.
Squared-error loss: Suppose your loss functions at each stage of the optimisation are:
$$\begin{equation} \begin{aligned}
\text{Loss}_1(\mathbf{X}) &= (\langle E \rangle_p - \langle \hat{E} \rangle_p)^2 + c \cdot n, \\[6pt]
\text{Loss}_2(\mathbf{X}') &= (\langle E \rangle_p' - \langle \hat{E} \rangle_p')^2 + c \cdot n'.
\end{aligned} \end{equation}$$
In these loss function we have squared-error loss in the estimand and we also impose a fixed cost of $c>0$ for each observation.  Your goal is to minimise the expected loss at each stage of your sampling, which yields the optimal sample sizes.
Solving the optimisation problem: Solving this kind of optimisation problem is quite a large exercise, and I will not undertake it here.  The first thing you would need to do is to determine the MAP estimators for the expected energy values under arbitrary sample sizes $n$ and $n'$.  Once you have found the form of the estimators, you will be in a position to find expressions for the expected loss at each stage of the sampling, and minimise the expected loss as a function of your sample size.
Since the second sample size $n'$ is chosen after observing the first sample $\mathbf{x}$ you will need to use backward-induction to determine the optimal sample sizes.  The process is as follows:


*

*Find (or assume) the form of the estimator $\langle \hat{E} \rangle_p (n,\mathbf{X})$.

*Find (or assume) the form of the estimator $\langle \hat{E} \rangle_p' (n',\mathbf{X'})$.

*Find the sampling function $\hat{n}'(n, \mathbf{x})$ that solves the second stage of the optimisation:
$$\text{Minimise} \quad F_2(n'|n, \mathbf{x}) \equiv \mathbb{E}(\text{Loss}_2(\mathbf{X}') | n, n', \mathbf{x}).$$

*Find the sampling function $\hat{n}$ that solves the first stage of the optimisation:
$$\text{Minimise} \quad F(n) \equiv \mathbb{E}(\text{Loss}_1(\mathbf{X}) + F_2(\hat{n}(n, \mathbf{X})|n, \mathbf{X}) ).$$
Note that the solution to the optimisation problem will depend heavily on the cost value $c$.  As $c \rightarrow 0$ the observations become costless and so the optimal sample sizes go to infinity, and as $c \rightarrow \infty$ the observations become infinitely costly and so the optimal sample sizes go to zero.  To get a meaningful optimisation you will need to focus suitable attention on the relative sizes of the cost of observations versus the loss from estimation error.
