Is it possible to use DDPG for discrete action space? In Deep Deterministic Policy Gradients(DDPG) method, we use two neural networks, one is Actor and the other is Critic.
From actor-network, we can directly map states to actions (the output of the network directly the output) instead of outputting the probability distribution across a discrete action space. It especially advantages in continuous action space problem so that most examples that I've found using a sigmoid function as the output activation function in Actor-network and multiply by action maximum bound.
However, my model has discrete actions (e.q. integer index [0-125]). In this case, how should I build the output layer of actor-network? should I also use a sigmoid function and just transfer it as an integer by brute-force?
 A: DDPG extends actor-critic methods from the discrete action-space environments they were originally developed on to continous action-space environments.
With that in mind -- sure, you can use actor-critic methods with discrete action-spaces, but it doesn't really make sense to talk about "DDPG" anymore.

In this case, how should I build the output layer of actor-network?

Typically for a discrete action, $\pi$ is bernoulli with $p$ parameterized by the output of the network.
A: I've struggled for a while with this same question. Actually, like shimao said, DDPG is the continuous action space version of actor-critic method. So for discrete action space, you may use DQN or Double-DQN instead.
A: In general, well you could. You can always write a code in your env.step(a) function that takes any continuous representation of your action and discretisze it. This modification is transparent to DDPG. As far the agent is considered, it just deals with a very peculiar kind of environment.
The question, is would it be helpful?
Answer: Depending on the representation of the action, it might help a lot. This is particularly useful if your action vector is very large but it factorizes.
As an example, imagine that you need to output an integer in [1, 2^K]. A discrete space treatment would require 2^K outputs which becomes prohibitly expensive even with moderate K values. However, you can re-structure your problem so that your DDPG's policy network outputs a K-sized vector with each element in (0,1), which is then discretized by the environment to {0,1}^K and then converted to the integer value.
Obviously, you could have many more factorizations (or econdings/decondings) of your action.
A: Actor critic methods wasn't originally developed for discrete action - spaces in fact it's exact opposite.
DDPG born from lack of training off policy on continuous action spaces  , since all proper actor critic algorithms are on policy and they have horrendous sample efficiency.
DDPG with discrete actions is basically DQN with improvements. Newer versions of DQN such as C51 and Rainbow nets are much more refined for your need , if you need discrete actions with off policy training.
