0
$\begingroup$

I am trying to implement $PPO$ for continuous action spaces so need probability of taking actions from a neural network with a tanh activation in the output layer since the action space ranges from $[-2, 2]$. I multiply the tanh output in the $[-1,1]$ range by $2$ to get this action.
I saw this: tanh converted to probability

but there is conflicting answers in the answer and comments:

The answer:

answer

0.5(a(x)+1) converts tanh to a probability.

Comments:

different formula

says it is $0.5(a(2x) + 1)$...which I don't really understand if $a(x)$ is the $\tanh$ output then am I supposed to feed $2x$ to the network rather than $1x$ just for the purposes of this conversion from $\tanh$ to probability?

I do not have enough rep to comment so I posted a new question here.

$\endgroup$
1
$\begingroup$

Both are fine. The idea is to the output of $a(x)=\tanh(x)$ to an increasing function that takes value from $(0,1)$.

Since for any real number $x$, $$-1 \le \tanh(x) \le 1$$

$$0\le \tanh(x) + 1\le 2$$

$$0\le \frac{\tanh(x) + 1}2\le 1$$

You can actually consider the map $\frac{a(tx)+1}2$ where $t$ is a parameter of your choice that control the variance of your distribution.

You might like to read about the logistic distribution where its CDF is

$$\frac12 + \frac12\tanh\left( \frac{x-\mu}{2s}\right)$$

where its mean is $\mu$ and its variance is $\frac{s^2\pi^2}{3}$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I just read the link for logistic distribution and saw that CDF is used in neural networks. I will try implementing it and the PDF of normal distribution to see which gives best results in the PPO algo. Do you actually recommend CDF more and if so why? $\endgroup$ – mLstudent33 May 26 '19 at 8:07
  • $\begingroup$ I am not familiar with PPO hence I can't answer the question. I think it depends on the property that you need in your problem, pdf can give you some values that are bigger than $1$ or not able to attain value of $1$ sometimes. $\endgroup$ – Siong Thye Goh May 26 '19 at 8:40
  • $\begingroup$ I see. I definitely do not want a value bigger than 1 but I think I am okay because I am using a range of two values to approximate the PDF at a given point along the distribution. $\endgroup$ – mLstudent33 May 26 '19 at 9:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.