I don't quite understand how a recurrent neural network or LSTM is trained for automatic speech transcription. Say I have n
audio files of speech, each with an associated transcript. I get that the neural network tries to predict the current phoneme, but how does the network know whether it was correct in guessing that phoneme or not? That phoneme occurred at a certain timestamp in the audio, does that mean that the transcript file needs to contain the timestamp of each phoneme for training?
UPDATE
Regarding the architecture of a recurrent neural network for this task, say the network's goal is to predict the word that was spoken. Does the the network just have a single output which is a word vector and then we train it based off of the ground-truth word vector? Is it possible to output after each frame the probability that that frame belongs to a certain phoneme? If so, how would one train the network without timestamp information?
For example, in this article, a predicted letter is outputted for each frame. Surely there wasn't any manual tagging for each frame for all the training files...