# Is value iteration considered a reinforcement learning algorithm or planning algorithm?

Recall value iteration:

$\text{Initialize$V_0(s) = 0 , \forall s \in S$} \\ \text{Repeat until convergence},\{\\ \quad \text{Given value function$V_i(s), s \in S$for iteration$i$do:} \\ \quad V_{i+1}(s) := max_{a \in A} \sum_{s'} T(s' | s, a)[ R(s,a,s') + \gamma V_i(s') ]\\ \}$

It seems that the algorithm assume that the reward and the transition probabilities are known. Hence, the environment is known and there really isn't any interaction with the environment (no samples from the environment), i.e. we get the model of the environment and compute the value function and the compute the optimal policy (if we want) as follows:

$$\pi(s) = argmax_a \sum_{s'} T(s' | s, a)[ R(s,a,s') + \gamma V_i(s') ]$$

Hence, it seems to me that the only "learning" done is of the policy (and value function). Hence, isn't this type of scheme better considered as planning rather than learning?

However, as Sutton points out, it is not necessarily helpful to make such a distinction in practice. For example, one can learn the state-value-function $Q(s,a)$ and use that one during action selection to plan (e.g. by using Heuristic Search).
To make the distinction clear, contrast this with Q-learning, a learning algorithm in which an agent acts on the basis of a current, possibly assumed $Q$ function, and then updates this function using value iteration after observing a reward.