# Sigmoid equivalent to Softmax exercise

I am currently studying the Sutton and Barto Intro To RL Book, and I'm trying to do exercise 2.9 (at the bottom of the following picture):

So the exercise wants me to show that the softmax is equivalent to the sigmoid and logistic function in the case when we have 2 actions.

I have seen this answer. I am going to try to replicate what he does:

Showing that $$\text{softmax}(x) \Leftrightarrow \sigma(x)$$

Let $$\mathbf{x}= \begin{pmatrix} H_t(a) \\ H_t(b) \end{pmatrix}$$. Then we can represent the softmax function as $$P(A_t=a)=\frac{e^{\beta_a H_t(a)}}{e^{\beta_a H_t(a)}+e^{\beta_b H_t(b)}}$$

The sigmoid we can represent the following way:

$$\sigma(x) = \frac{1}{1+e^{-\beta \mathbf{x}}}$$ and we make $$\beta = \begin{pmatrix}\beta_a \\ -\beta_b\end{pmatrix}$$ so that $$\sigma(x) = \frac{1}{1+e^{-\beta_aH_t(a)+\beta_b H_t(b)}}=\frac{e^{\beta_aH_t(a)}}{e^{\beta_aH_t(a)}+e^{\beta_bH_t(b)}}$$

But then how do I get rid of the $$\beta$$? It doesn't seem like I proved that they are equivalent. Any help?

$$\sigma(x) = \frac{1}{1 + e^{-x}} = \frac{e^x}{e^x + 1}$$
so you don't need any $$\beta$$'s. The $$\beta$$'s appear in logistic regression. See also the Softmax vs Sigmoid function in Logistic classifier? thread. So you've proven what you wanted to show.
• Why don't I need $\beta$'s? I don't see how I can prove it without including them. I mean if $x=\begin{pmatrix} H_t(a) \\ H_t(b) \end{pmatrix}$ then how is $p(A_t=a)=\text{softmax}(x)=\frac{e^{H_t(a)}}{e^{H_t(a)}+e^{(H_t(a)}}=\sigma(x) = \frac{e^{\begin{pmatrix} H_t(a) \\ H_t(b) \end{pmatrix}}}{e^{\begin{pmatrix} H_t(a) \\ H_t(b) \end{pmatrix}} + 1}$? I actually referenced the question you reference in your answer! Commented Jan 20, 2022 at 9:39
• Actually that's not the correct answer I think. They can't make $H_t(a) \leftarrow H_t(a)$ and $H_t(b)\leftarrow 0$ Commented Jan 20, 2022 at 10:23