0
$\begingroup$

Do you know or have heard about any cutting edge deep reinforcement-learning algorithm which can be successfully applied for discrete action-spaces in multi-agent settings?

I have been researching and I have found MADDPG and Soft Q-learning algorithms as the top ones in the state-of-the-art. They are mainly focused on environments with continuous action space. Although they can be applied to discrete action-space (e.g. MADDPG with gumbel softmax) it seems it is not what they are intended for (I have tried with MADDPG (w/ Gumbel softmax) achieving disastrous results...). In their corresponding papers they don't give a lot of details of how to use them in these settings.

$\endgroup$
0
$\begingroup$

Actually, you don't need to use gumbel softmax for discrete actions. The output of actor network give you probability for each action(for discrete action space), by applying a constraint that a_t ~ [0,1]. That can solve your problem.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Do you mean using a softmax in the output of the actor? I tried it but it doesn't work. I think the problem with this approach is that when you feed the critic in with the output of the softmax to calculate the gradient of the actor, as this output doesn't include the "sampling" factor, it just doesn't propagate the desired gradient... wdyt? $\endgroup$ – ivallesp Dec 29 '18 at 23:53
  • $\begingroup$ Your intuition can be right, while using softmax. But i haven't seen anyone using softmax at actor's output. Give it a try, without softmax. $\endgroup$ – Ankish Bansal Dec 31 '18 at 3:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.