# Risk-averse multi-armed bandits

We want to pose one problem as a multi-armed bandit setting. The issue is that some of the arms are very risky with potentially undesirable effects (or not). Is there a way to do a risk-aware exploration where you stop OR reduce exploration of particular arm(s) if the rewards observed are of concern while still exploring & exploiting other 'safe' arms?

I can think of the softmax multi-armed bandits, but maybe there is something smarter than that.

Consider this simple example. Say, you want the agent to avoid all options that can lead to negative rewards. If you have access to the distributions of values for each arm, you can map the value distributions to a (scalar) metric that is not the expected reward, but rather: $$V(x_i) \rightarrow -\infty$$ if $$\int_{-\infty}^{0} p(V(x_i)) dV > \epsilon$$, else $$V(x_i) = \int V(x_i) p(V(x_i)) dV = mean(V(x_i))$$, where $$\epsilon$$ is an arbitrarily small number. In words, for all options that have negative rewards, you artificially set the scalar value which you will use to choose an action to $$-\infty$$ while using the mean value for all other options that do not have negative rewards.