I've asked this question on cs.stackexchange before. It has a 20-hours remaining bounty there.

On-line handwriting recognition is the task of converting a series of $(x(t),y(t))$ coordinates to symbols and words. In contrast to off-line handwriting recognition, where you only have a bitmap of the written symbols, you have the way it was written in on-line recognition.

The paper Npen++: An On-Line Handwriting Recognition System describes a system that makes use of Hidden Markov Models (HMMs) and Artificial Neural Networks (ANNs) to recognize symbols.

[...] each character is modelled with three states representing the first, middle and last part of the character. Hence, the MS-TDNN can be regarded as a hybrid recognizer that combines features of neural networks and hidden Markov models (HMMs).

Source: Npen++: An On-Line Handwriting Recognition System, page 6.

One of my problems is that I don't know what a TDNN (Time delay neural network) is. At the moment, I think I've understood what feed-forward neural networks are and I know that they are trained with backpropagation. But all examples I have seen so far have a fixed input vector (e.g. colors of a fixed-size image; positions on a fixed-size board).

How exactly are the lists of coordinates mapped to the input neurons?

Is the following "mind model" I have of those HMMs correct?

enter image description here

When I understand it correct, then you have one HMM for each symbol. How do you get the probabilities $p_1, p_2, p_3, p_{1,2}, p_{2,3}$? How do you get the probability which character to recognize given the trained HMMs and a list of coordinates? When you can get a probability for every symbol with HMMs only, how does the ANN help?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.