# Why use $-\dim(|A|)$ as target entropy in soft actor critic

I am trying to implement Soft Actor-Critic with automatic Entropy tuning.

One thing I noticed is that the authors, and also the majority of implementations of SAC use $$-\text{dim}(|\mathcal{A}|)$$ as entropy target $$\bar{\mathcal{H}}$$ in the loss function of $$\alpha$$, and I am confused where this exact value came from. I have read the answer to this question, but I am still not fully convinced.

I do understand the intuition behind setting $$\bar{\mathcal{H}}$$ proportional to $$\text{dim}(|\mathcal{A}|)$$. Let's say, for example, we consider the target distribution of $$\text{tanh}(\bf{u})$$ to be $$\text{Uniform}(-1, 1)$$ for each dimension (I am not sure if this is a good assumption though), then the entropy target should be $$\text{dim}(|\mathcal{A}|) \ln(2)$$. But I am not sure why the majority of the implement uses exactly $$-\text{dim}(|\mathcal{A}|)$$.

Is there any specific reason for this setting? Or, are there any papers or articles about how the target entropy setting affects the performance of SAC?

Edit:

Just in case, the loss function of $$\alpha$$ I am referring to is Eq18:

$$J(\alpha) = \mathbb{E}_{\bf{a}_t \sim \pi_t} [- \alpha \log \pi_t(\bf{a}_t | \bf{s}_t) - \alpha \bar{\mathcal{H}}]$$

If you are minimizing a loss function $$f$$, and there is a function $$g$$ that is proportional to $$f$$ something like this
$$g = k \cdot f$$ then by minimizing $$f$$ you are also minimizing $$g$$.

It also extends to all transformations of f that do not change the ordering. If for a transformation $$h(f)$$ if you were to sort both $$f$$ and $$h(f)$$ and the order of the result is identical, then minimizing on $$f$$ is the same as minimizing on $$h(f)$$. Its said a little awkwardly, sorry.

Positive examples:

lets say that $$f=\operatorname{abs}(x)$$, you could minimize $$h(f)=f^2$$ and get the minimum would be the same.

lets say that $$f=\operatorname{abs}(x)$$ and $$x$$ is constrained to $$0. you could minimize $$h(f)=\log(f)$$ and get to the same location.

Negative examples (how to do it wrong):

lets say that $$f=\operatorname{abs}(x)$$ and $$x$$ is NOT constrained to $$0. you could minimize $$h(f)=\log(f)$$ and you would run into the discontinuity at $$x=0$$ and your optimization would fail.

• Can you elaborate on how this is exactly connected to the selection of entropy target $\hat{\mathcal{H}}$? The explanation is to vague and I really do not see the connection with the loss function in this answer and the loss function used for adjusting $\alpha$. Commented Jun 1, 2023 at 14:23
• Where it says "then the entropy target should be $dim(|A|)ln(2)$" you are differing about a constant of proportionality. I will check the paper, but some folks maximize the negative loss and others minimize the negative gain. Signs can get lost, but inverting during minimize can be a very bad thing. It might be a typo. Let me check. Commented Jun 1, 2023 at 16:28
• The $\text{dim}(|\mathcal{A}|) \ln (2)$ part was just an example that is not even sure if it is valid or not. Also, I wouldn't say the loss function for $\alpha$ used in the paper is proportional to $\bar{\mathcal{H}}$. Commented Jun 5, 2023 at 5:26