I'm not sure if you personally have resolved the issue yet, but I've just read the same paper and perhaps my understanding of the soft-window mechanism can be of assistance to anyone else that stumbles upon this question.
So, on with the answer: Graves in particular notes that the transducer RNN approach (which you can find in another paper of his) does not seem to perform very well when applied to handwriting synthesis, so another approach is used --- this is the origin of the synthesis network approach that is presented in section 5, "Handwriting Synthesis".
But before we talk about the new soft-window-based RNN approach, let's talk about what it means, abstractly, to perform handwriting synthesis: mathematically speaking, you're given the ability to look at an entire sequence of characters, and you're tasked with the problem of generating a sequence of $\mathbb{R}$-valued vectors; notably, you want to avoid having to think too hard about alignment between the characters and the real vectors, because this seems like a hard thing to model by hand.
So regarding the handwriting synthesis problem, you can think of it as taking a sequence
$\vec{c} = (c_1,c_2,\ldots,c_U)$
of characters (which you can think of as one-hot-encoded vectors $c_i \in \{0,1\}^S$ where $S$ is your alphabet size) that you have access to, and you need to generate a sequence
$\vec{x} = (x_1,x_2,\ldots,x_T)$
of real vectors one at a time, representing the computer-synthesized/handwritten text thus far.
One thing comes to mind: how do you get this sequence $\vec{x}$ when you start from nothing but $\vec{c}$? Well, let's assume that you already have the whole sequence minus the last vector:
$\vec{x}_{[1:T-1)]} = (x_1,\ldots,x_{T-1})$
and assume that you already have a network model $\mathcal{N}$ that models the probability distribution
$p(x_t | x_1,\ldots,x_{t-1}; \vec{c})$;
then you can generate $x_t$ by making a prediction based on this distribution.
Hence, thinking recursively, you can generate the whole sequence just by feeding the zero vector $\vec{0}$ at time $t=0$ to get a distribution for $x_1$, which you can feed back into the model and get a distribution for $x_2$, and then you can keep doing that over and over again until you generate the whole sequence $\vec{x}$. You don't know how many times you will need to keep doing this, though, which is why the RNN comes into play here.
All of this is just the background on the purpose of prediction RNNs.
Now, moving on to the soft window: what's the intuition behind it? The paper gives the following definition of the window: at a time step $t$ (i.e. when you're in the process of generating the $t$-th vector/penstroke), it is defined as
$
w_t := \sum_{u=1}^U \phi_t(u)c_u,
$
where I've put the $t$ in the subscript to emphasize the fact that you have a single $\phi(u)$ for each timestep $t$. But this $w_t$ is just a weighted sum over your sequence of characters! So think of it this way: it's a "fuzzy" way of looking at a particular spot in your sentence $\vec{c}$ --- not only looking at a particular character in the sequence of characters, but "smearing" your attention to the characters to the left and right of it, where the "smearing" $\phi_t$ varies depending on what time $t$ it currently is --- which makes sense, because you should be focusing on the next part of the sentence as you write more and more text, right?
Now how do you get this attention-smear $\phi_t$? Well, it's defined as a mixture of Gaussians functions
$\phi_t(u) := \sum_{k=1}^K \alpha_{k,t} \exp(-\beta_{k,t} (\gamma_{k,t} - u)^2)$,
which is just a fancy way of saying that it's a weighted sum of a handful of functions that look like smooth peaks over the character sequence $\vec{c}$. (In the definition above, I rearranged some of the subscripts and variable names to make it look less confusing to me, but nothing essential has been changed.)
This looks like a complicated expression, but it's just saying that $\phi_t$ is a set of weights given by adding a bunch of "bump"-looking functions that are all clustered "near" $c_u$. As for just exactly how close to $c_u$ the weights are, how spread out they are, etc., that's learned by transforming the output of the first hidden layer.
It's helpful to see what this means in a small test case. So let
$\vec{c} = (c_1,c_2,c_3)$
where
$c_1 = (1,0,0)', c_2 = (0,1,0)', c_3 = (0,0,1)'$
are one-of-3-encoded characters, and suppose at some time $t$ your soft window model tells you that your $\phi_t(u)$'s are
$\phi_t(1) = 0.1, \phi_t(2) = 0.8, \phi_t(3) = 0.1$.
Then your soft window at time $t$ is
$w_t = \sum_u \phi_t(u)c_u = (0.1,0.8,0.1)'$.
So we can see that this particular setting of $\phi_t$ gives $c_2$ the lion's share of the influence, while allowing $c_1$ and $c_3$ to have a little bit of influence on the outcome as well.
The "big picture" structure of the synthesis network is composed of the following layers, in this order (see page 27 of the paper):
An input layer that consists of the predicted real vector $x_t$ representing a penstroke, informed by the previous output of the network $y_{t-1}$;
a (recurrent) hidden layer $h^1_t$ that learns the weights to the soft window, which as we've mentioned can be thought of as a sequential form of "fuzzy attention" that slides down the text input $\vec{c}$ --- note that $h^1_t$ has knowledge of its previous state $h^1_{t-1}$ as well as the previous window information $w_{t-1}$ and the penstroke $x_t$ that was just predicted;
a soft window layer $w$ that receives input from the previous hidden layer $h^1$ as well as the character sequence $\vec{c}$ and feeds its learned representation to the second hidden layer $h^2$;
the second (recurrent) hidden layer $h^2$ gets its information from the soft window layer $w$ as well as from $\vec{x}$, the input sequence generated thus far; this latter part is important because it allows information from the input sequence to directly influence the second hidden layer.
an output layer returning a representation of the probability distribution over the next predicted penstroke.
Phew, that took a while. If anyone has any corrections, I'm open to hearing them :-) This is my intuition of the architecture of the synthesis network model in the paper. Hopefully someone else can add onto it!
