Thanks. You are right about the infinite horizon case, it was my confusion. So, theoretically, if we have a stationary, infinite-horizon MDP, is the optimal policy stationary or it might be non-stationary?

Example for non-stationary optimal policy: MDP with 0 stage rewards and 2 terminal states, $s_1$ and $s_{10}$ with $r(s_1)=1, r(s_{10})=10$. Consider starting at $s_0$ which is right next to $s_1$ and 5 steps away from $s_{10}$. Let $T=5$. Initially, you will try to go to $s_{10}$. Since the environment is stochastic, you might end up again at $s_0$ at step 4. As you know you cannot reach $s_{10}$ in the remaining steps, you will try to go to $s_1$ - hence the choice depends on t. Similarly, with some good value of the discount factor, the same case can be made in the infinite horizon setting.

Stationary means time-invariant. The equation follows directly. You can have a look at Wikipedia en.wikipedia.org/wiki/Stationary_process. The definition can also be found in any decent book. The fact that the optimal policy for the finite horizon setting is not guaranteed to be stationary is mentioned here: goo.gl/52HGjg (around 15:25). I actually never found a formal statement that the policy in the infinite horizon setting is stationary. I only inferred this because all major RL literature uses this assumption (for example I have never seen $Q(s,a)$ which depends on time).

Thanks, I did not clarify my initial question well enough. Please see the edit.