I am reading sutton's reinforcement learning an introduction. The book mentioned that for continuous state space, one could leverage value function with the form

V(s, w) = W*S,

where s represents state and w the weights. So if there is only 1 factor in a state, the function could be like, for instance,

V(s, w) = w_1*s + w_0

However, what if the state has 2 or more components? Say S = S(position, time), then how to rewrite the above function?

Is it V(s, w) = w_1*(position + time) + w_0?

Thanks in advance


1 Answer 1


Typically each state represented by a vector in $\mathbb{R}^d$ (likewise for actions).

Then one can write $\hat V(s,\theta) = w^Ts + b$ where $s,w \in \mathbb{R}^2$, $b \in \mathbb{R}$.

Of course this is not really an adequate parameterization of value functions, but just a linear estimator of it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.