# How to stabilize training in multi-objective reinforcement learning?

I am trying to train an agent to maximize multi-objectives. If I just add up rewards from different objectives, my problem is that the agent maximizes the 'easy' objective, at the expense of the hard one. How can I adaptively penalize the easiness with which the agent realize an objective?

If the reward from each objective $1... k$ is $r_1, r_2, ... r_k$ set the goal to be maximization of $R = \min_i r_i$ instead of $R = \sum_i r_i$. By maximizing the minimum reward across all goals, the agent will be forced to learn all objectives in order to perform better.
A possibly better behaved alternative would be to set $R = -|\text{softmax}(-\vec r)|_{L_1}$ (this sets the reward to be the negative softmax of the cost). This should accomplish about the same thing while being a bit smoother.