Timeline for Natural way to construct stochastic policy from value function?

9 events

when toggle format	what		by	license	comment
Mar 20, 2019 at 23:27	vote	accept	Kris
Mar 20, 2019 at 23:27	answer	added	Kris		timeline score: 0
Feb 1, 2019 at 20:24	comment	added	Kris		@nbro Right okay. Let's narrow down the question. Is there a preferred way of mapping $\mathbb{R}$ to $\mathbb{R}_+$ e.g. taking the absolute value or taking a square, or is exp somehow preferred? I'm guessing the exp has nice properties for normally distributed values.
Feb 1, 2019 at 19:50	comment	added	Kris		@nbro Indeed, intuitively you want to select the action with higher expected return. It cannot be proportional to the return, however, because returns typically don't live in the same space as probabilities. For instance, for real-valued returns you need to first map the Q values to the upper half-space and then normalize. This can be done in countless ways, of which softmax is just one specific choice.
Feb 1, 2019 at 19:17	comment	added	shimao		@nbro i didn't make myself clear -- i'm talking about a hypothetical environment with one non-terminal state and two actions from that state.
Feb 1, 2019 at 18:32	comment	added	shimao		@nbro no. if one action gives 1 reward and the other gives 2, the optimal policy does not select the worse action 1/3rd of the time. the optimal policy always goes for the 2-reward action.
Feb 1, 2019 at 2:55	history	edited	Kris	CC BY-SA 4.0	added note on working example for Expected-SARSA
Jan 31, 2019 at 6:50	review	First posts
Jan 31, 2019 at 9:07
Jan 31, 2019 at 6:47	history	asked	Kris	CC BY-SA 4.0