Note that $(y_j−\hat{y}_j)$ is 0 if the sample is at the correct side of the decision boundary. So there are only updates when the estimate is wrong.
Otherwise, let's assume $y_j=1$ and and we estimate the wrong label: $\hat{y}_j = -1$
The update becomes $\textbf{w}_{t+1} = \textbf{w}_{t} +\eta \textbf{x}_j$. That is, we move the direction of the vector $\textbf{w}$, which is perpendicular to the decision boundary, slightly towards the misclassified data point.
Note that $\textbf{w}_{t+1} \cdot \textbf{x}_j = (\textbf{w}_{t} + \eta \textbf{x}_j) \cdot \textbf{x}_j = \textbf{w}_{t} \cdot \textbf{x}_j + \eta \textbf{x}_j \cdot \textbf{x}_j = \textbf{w}_{t} \cdot \textbf{x}_j + \eta \Vert\textbf{x}_j \Vert > \textbf{w}_{t} \cdot \textbf{x}_j$
since $\eta \Vert\textbf{x}_j \Vert$ is positive (unless $\textbf{x}_j=\textbf{0}$).
Since the length of all samples is constant, every update is a finite update, although relative to the size of $\textbf{w}$ they will become smaller, and hence the angles will update with smaller steps eventually.
A symmetrical argument holds for $-1$ samples.
Since $\hat{y}_j = sign(\textbf{w} \cdot \textbf{x}_j)$, this means ultimately all estimates are correct when the data is linearly separable.
