Why do we use expectation in reinforcement learning? In reinforcement learning we use lot of expectations, what's the logic behind them?
Like in reward function, we use expected return instead of return, in value functions V and action-value functions Q we use expectation. Almost every-term in reinforcement learning is written in the form of expectation. Why?
Why do we use expectations?
 A: Because life is uncertain.  We don't know what the future might hold.
If we knew the future, we'd calculate the reward we'll receive for each possible action, and choose the best one -- but alas, no one can tell the future.  Therefore, we can't be sure what the reward of each possible action will be, and we can't be sure which action is best.
So, instead, for each action, we calculate the average of all possible rewards (weighted by their likelihood).  Roughly speaking, this is our best guess at what the reward of the future might be, given the information available to us right now and the unavoidable uncertainty about the future.  Then, we use that information to guide our decisions.
A: You can view this as a consequence of how we define "optimal" in most reinforcement learning applications: An optimal policy is that which maximizes expected discounted reward in a Markov decision process. MDPs are RL's core and longest-studied problem, making them a natural starting point.
Though natural, this definition may not fit every application. Generalized MDPs replace the $\max\limits_a$ and $\mathop{\mathbb{E}}\limits_{s'}$ operators with other non-expansions. For example, replace $\mathop{\mathbb{E}}\limits_{s'}$ with $\min\limits_{s'}$, and you have a risk-sensitive MDP. 
Several standard planning and learning algorithms—value iteration, policy iteration, model-based RL and Q learning—can be generalized to work in this framework. (Szepesvári and Littman.)
A: We use expectations because we want to optimize the long-term performance of our algorithms. This is the weighted sum of all possible outcomes multiplied by their probabilities — the expected reward.
