In Reinforcement Learning (DQN), is there a way to constrain/penalise the model so that it doesn't take a different action very often? The RL model I am building is one form of DQN. Its internal network is a regular deep NN.
In the application I am looking at, there is a cost for taking a different action (compared to the previous action).
That means, even when the model achieves a positive reward r when going from t to t+1, this r would be reduced by the cost if the action taken is different from the last action. Currently the model inputs are the environment data.
Is there any way to penalise the model for choosing to take different actions often? Or any other form of constraints that I could apply to the model?
 A: As I understand your description, the previous action taken influences the distribution of reward values possible from the current time step. That means effectively the previous action should be part of the current state in order to maintain the Markov property - which is theoretically required so that the agent can correctly assign expected returns to action values.
Add the previous action taken to the state features - for a neural network approximator in DQN, I would expect that to be a one-hot-encoded feature, i.e. multiple new input fields.
This will correctly model the MDP as you need it to describe your problem. Everything else should then fall into place then with no need to change how the DQN solver works.
A: If your aim is to "penalize the model for choosing to take different actions often", one possibility is to penalize the entropy of the policy on top of the usual expected future reward with some tradeoff coefficient $\lambda > 0$:
$$ E_{\pi}\left[ \sum_{t=0}^T \mathcal{R}(s_t,s_{t+1},a_t) - \lambda H(\pi(a \vert s_t)\right]$$
Note this is the opposite of what is commonly done to avoid collapse of the policy to greedy behavior, and you should be very careful with this.
