It's untrue that $cov(y,z) = 0$ whenever $z$ does not directly affect $y$. The key is the word "directly". We hope to find an instrument such that "z influences x which influences y". This would mean that $cov(y,z|x) = 0$ (that is, if we observe the value of $x$, knowing $z$ tells us nothing incremental about $y$), but it does not mean that $cov(y,z) = 0$ unconditional on $x$.
The distinction you need to make here is the difference between an associative (or correlative) effect and a causal one.
Let's first look at an example (linked at bottom). Suppose you want to know how the size of a political protest ($x$) causally affects local political outcomes ($y$).
However, if you simply look at the association between $x$ and $y$, you may note a number of potential issues. For example, the size of the protest ($x$) could itself be correlated with other factors like the importance of the issue or the likelihood of the protest causing change. These factors are also correlated with $y$. Hence, they confound the relationship between $x$ and $y$.
So how can an instrument variable help us? Consider the instrument of how good the weather was on the day of the protest ($z$). The intuition is that weather ($z$) could directly impact crowd size ($x$); however weather on the day of a protest ($z$) should not directly influence a local political outcome ($y$). The only way you would expect $z$ and $y$ to be related is through $x$.
Example Source: https://dash.harvard.edu/bitstream/handle/1/13457753/TeaParty_Protests.pdf