I want know how to add a constraint to Q-learning. I have an action resulting in two rewards every time (reward 1= delivery cost , reward 2= delivery time). I want to minimize the cost while ensuring max delivery time limit is not violated. Is there a standard/formalized way to do this?
Is there a standard/formalized way to do this?
Yes. There is no way to optimise across two independent variables, they must be combined into a single metric. This is typically done using a linear sum. If one value must be minimised and the other maximised, then typically in RL you express the value to be minimised as a negative reward value, because RL usually maximises reward (this is not a requirement, you can easily re-write any RL algorithm to minimise a long-term cost, just the literature mostly talks about maximising a long-term reward).
So you must combine the two metrics, decide the weights for the disparate rewards, perhaps based on monetary business value. Then you add them together. If there are strict constraints, either ensure action or state space does not allow the possibility of them, or if that is not feasible, then penalise an unacceptable result with a large negative reward. There is no requirement that you scale rewards linearly.