'undirected' $\epsilon$-greedy action selection There are two famous/classical ways to select an action under the $\epsilon$-greedy action selection that discusses the trade off between exploration and exploitation. 
Firs is the Semi-uniform Random Exploration. In this case, the best action is selection with some probability $p$, while a random action is chosen with probability $1-p$. 
Second is the Boltzmann exploration, whose function is based on the Boltzmann distribution of statistical mechanics
$$P(a) = \dfrac{exp(Q(s,a)/T)}{\sum_a exp(Q(s,a)/T)} $$
My question is this: 
The paper: Bayesian Q Learning by Dearden, Friedman and Russell, calls both these exploration methods 'undirected'. Meaning, there is no exploration-specific knowledge used. 
I do not understand why it is 'undirected'. Further, the very short explanation they gave is not intuitive, either. 
Any insights? Any possible reason why this is undirected, and what examples are 'directed'? 
Thanks
 A: As the paper puts it, "Most of the directed techniques can be thought of as selecting an action to perform based on the expected value of the action plus some exploration bonus[11]." 
If you follow that citation, you'll find this paper, which gives these definitions:

  
*
  
*undirected techniques that do not use any "exploration-specific" knowledge about the learning process, 
  
*directed techniques that remember knowledge about the learning process and use it to direct exploration.
  

To get an idea what "knowledge about the learning process" might mean, see this snippet:

The exploration bonus $\delta_k$ represents the maximum amount (of reward or utility) that one is willing to pay for one observation of the output of arm $k$ (fig. 1). It measures the importance of sampling this arm to obtain information rather than simply obtaining the expected reward $\bar\rho_k$. [...] This reflects the fact that, when an arm has been tried an infinite number of times, its characteristics are known with certainty and thus, there is no further information
  to be learned from sampling it.

In other words, directed methods account for uncertainty, and formalize an agent's preference for exploration, in the form of a bonus function. This has the effect of directing an agent to explore actions of uncertain utility. 
