If the state set $\mathcal{S}$ and action set $\mathcal{A}$ of a Markov Decision Process are infinite does an optimal value function $v_\pi(s)$ exist?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Not necessarily. Consider an environment with a single state and a countably infinite set of actions. Enumerate the actions with the bijection $f: \mathcal{A} \rightarrow \mathbb{N}$. Suppose the reward for taking a given action $a$ is $f(a)$. Then since there is no greatest natural number, there is no optimal policy, and therefore no optimal value function.
-
$\begingroup$ just saw that title says "continuous" -- in which case substitute $\mathbb{R}$ for $\mathbb{N}$ in above answer $\endgroup$– shimaoCommented Mar 4, 2020 at 8:50
-
$\begingroup$ Thanks, I don't think I phrased my question correctly but that still answered the question I had in mind. $\endgroup$– KaneMCommented Mar 4, 2020 at 10:30