In the second edition of the book "Reinforcement Learning: an introduction" by Sutton and Bato page 324 (Policy gradient chapter): It says that: Given a state, the effect of the policy parameter on the actions, and thus on reward, can be computed in a relatively straightforward way from knowledge of the parameterization. But the effect of the policy on the state distribution is a function of the environment and is typically unknown.
Can anyone explain why it is straightforward for actions and why it is unknown for state distribution? Thank you.