As far as Im aware, the difference between Multi-armed Bandit problems and full MDPs is that in MABs the full distribution over the results of action are known. Is this true?
It is untrue.
The MAB setting can be interpreted as a one-state MDP (thus a MAB is stateless). In MAB you are given a set of K distributions (whose structure is usually known while the mean of the distribution is unknown).
The goal in MAB is either to identify the arm with the highest mean (best arm) with the smallest number of arm pulls or to maximize the cumulative regret.
An MDP is much more complex: the action you choose determines a new state in which you could have completely different distributions (and best arms). In MAB you are always in the same state).