# How can you use HMMs and ANNs for on-line handwriting recognition?

I've asked this question on cs.stackexchange before. It has a 20-hours remaining bounty there.

On-line handwriting recognition is the task of converting a series of $(x(t),y(t))$ coordinates to symbols and words. In contrast to off-line handwriting recognition, where you only have a bitmap of the written symbols, you have the way it was written in on-line recognition.

The paper Npen++: An On-Line Handwriting Recognition System describes a system that makes use of Hidden Markov Models (HMMs) and Artificial Neural Networks (ANNs) to recognize symbols.

[...] each character is modelled with three states representing the first, middle and last part of the character. Hence, the MS-TDNN can be regarded as a hybrid recognizer that combines features of neural networks and hidden Markov models (HMMs).

Source: Npen++: An On-Line Handwriting Recognition System, page 6.

One of my problems is that I don't know what a TDNN (Time delay neural network) is. At the moment, I think I've understood what feed-forward neural networks are and I know that they are trained with backpropagation. But all examples I have seen so far have a fixed input vector (e.g. colors of a fixed-size image; positions on a fixed-size board).

How exactly are the lists of coordinates mapped to the input neurons?

Is the following "mind model" I have of those HMMs correct?

When I understand it correct, then you have one HMM for each symbol. How do you get the probabilities $p_1, p_2, p_3, p_{1,2}, p_{2,3}$? How do you get the probability which character to recognize given the trained HMMs and a list of coordinates? When you can get a probability for every symbol with HMMs only, how does the ANN help?