Choosing "Target Entropy" for Soft-Actor-Critic (SAC) algorithm I am quite familiar with Soft-Actor-Critic (SAC) and its many applications in continuous control RL environments. However, when implementing this algorithm in a practical setting, one thing that still eludes me is how the authors choose the "target entropy"  hyperparameter (denoted $\bar{\mathcal{H}}$) which is used to automatically tune the entropy temperature parameter $\alpha$ during training. Classically, they choose this target entropy to be $\bar{\mathcal{H}}=-\dim\lvert\mathcal{A}\rvert$ where $\mathcal{A}$ is the action space, but I don't know why the authors chose this value. Does anyone have some intuition as to where this choice came from/when other choices might be appropriate? I have looked around at numerous SAC related papers to see if the effect of this target entropy is measured, but so far my efforts have been unsuccessful.
 A: I can't answer on behalf of the authors, but it makes sense to me that they would choose a default value $\propto \text{dim}(\mathcal{A})$. If they instead chose it at some fixed (i.e. not dependent on action dimensionality) constant, then problems with larger action dimensionalities would have to spread the same budget of randomness over the different action dimensionalities. For very large action spaces this might result in effectively deterministic behaviour. By choosing $\bar{\mathcal{H}} \propto \text{dim}(\mathcal{A})$, they are making sure that if you double the action dimensionality then you also double the amount of "total" randomness allowed by the policy, where "total" randomness is differential entropy (if you are confused why they have chosen a negative constant of proportionality, remember that differential entropy can be negative, and that differential entropies add across dimensions for diagonal stochastic policies; I think it just happens to be the case that for most mujoco tasks, it's effective to have relatively low target entropies. This might not hold for non-mujoco tasks.)
