Original paper: https://arxiv.org/pdf/1703.03864.pdf

On page 2-3, it writes,

enter image description here

But this equation is clearly wrong.

Since the step needed to derive the equation involves the argument, let $$\theta = \psi + \sigma \epsilon, \epsilon \sim N(0,I)$$

Therefore by a straight forward substitution, $$\mathbb{E}_{\theta \sim p_\psi} F(\theta) = \mathbb{E}_{\epsilon \sim N(0,I)} F(\psi + \sigma\epsilon) $$

and NOT,

$$\mathbb{E}_{\theta \sim p_\psi} F(\theta) =\mathbb{E}_{\epsilon \sim N(0,I)} F(\theta + \sigma\epsilon) $$ ($\theta$ and $\psi$ exchanged) as written in the paper. Hence all the rest of the results are wrong.

One can argue that the authors, confusingly, implicitly exchanged the notation of $\psi$ and $\theta$, where now $\theta$ refers to the mean of the Gaussian distribution (which was previously denoted by $\psi$)

However, this implies that their main algorithm is wrong, enter image description here

Because $\theta$ here is clearly not the mean of the Gaussian. $\theta$ clearly refers to the neural network parameter (weights) in the original paper.

Can someone check for me?

Note that this error is also pointed out in an earlier post How is the equation in "Evolution Strategies as a Scalable Alternative to Reinforcement Learning" derived? however, in my opinion this is not just an abuse of notation "confusing" but it is flat out wrong.

  • 2
    $\begingroup$ Does this answer your question? How is the equation in "Evolution Strategies as a Scalable Alternative to Reinforcement Learning" derived? $\endgroup$
    – Xi'an
    Commented Jan 4, 2020 at 8:20
  • $\begingroup$ @Xi'an No. The answer said "$\theta$ to be the "mean parameter vector". But as I have pointed out in my question, the way that $\theta$ is updated in algorithm 1 does not imply that $\theta$ is the mean parameter vector, but the original weights of the neural network. Unless, we take the weights of the neural network to be the mean of a Gaussian....which the paper does not claim. $\endgroup$
    – Olórin
    Commented Jan 4, 2020 at 20:35

1 Answer 1

  • For the first question, I think the authors just exchange the notations $\psi$ and $\theta$.
  • For the second question, the weights of neural networks are modeled as Gaussian variables with distribution $\mathcal{N}(\theta, \sigma^2I)$.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.