12
$\begingroup$

I am reading this paper: skype translator where they use CD-DNN-HMMs (Context dependent Deep neural Networks with Hidden Markov Models). I can understand the idea of the project and the architecture they've designed but I don't get what are the senones. I have been looking for a definition but I haven't found anything

—We propose a novel context-dependent (CD) model for large-vocabulary speech recognition (LVSR) that leverages recent advances in using deep belief networks for phone recognition. We describe a pre-trained deep neural network hidden Markov model (DNN-HMM) hybrid architecture that trains the DNN to produce a distribution over senones (tied triphone states) as its output

Please if you could give me an explanation about this I would really appreciate it.

EDIT:

I've found this definition in this paper:

We propose to model subphonetic events with Markov states and treat the state in phonetic hidden Markov models as our basic subphonetic unit -- senone. A word model is a concatenation of state-dependent senones and senones can be shared across different word models.

I guess they are used in the Hidden Markov Model part of the architecture in the first paper. Are they the states of the HMM? The outputs of the DNN?

$\endgroup$
3
  • $\begingroup$ A senone is a speech recognition term. Is that the definition you're looking for, or clarification of how they're modeled in that paper? $\endgroup$ Dec 2, 2015 at 13:15
  • $\begingroup$ More like the application of then to the DNN-HMM. They are the states in the HMM but also de outputs of the DNN ? $\endgroup$
    – davidivad
    Dec 2, 2015 at 16:50
  • 1
    $\begingroup$ This article describes senones in some detail ... cmusphinx.sourceforge.net/wiki/tutorialconcepts $\endgroup$ Dec 31, 2016 at 20:43

3 Answers 3

9
$\begingroup$

"Senones" was named by me in 1992. See my ICASSP 1992 paper[1]. It's just a fancy name for a cluster of shared Markov states, representing similar acoustic events. It came from the contrast with IBM's fenones, where the "f" means "frame" and my "s" means "state".

The initial idea came from my 1991 Eurospeech (now called Interspeech) work, where I used top-down clustering on Markov states. You can find my CMU tech report in 1991 here: https://www.semanticscholar.org/paper/Shared-distribution-hidden-Markov-models-for-speech-Hwang-Huang/33ea989f1655636162b7e9b8e0cfe3fcce92c37d

In 1992, I decided to move it to tree clustering so that one could model unseen cd phones as well.

[1] Hwang, Mei-Yuh, and Xuedong Huang. "Subphonetic modeling with Markov states-Senone." In [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, pp. 33-36. IEEE, 1992. link: https://ieeexplore.ieee.org/document/225979

$\endgroup$
1
  • 1
    $\begingroup$ please add full reference for your link in case it dies in the future $\endgroup$
    – Antoine
    Mar 4, 2020 at 9:40
7
$\begingroup$

This is what I've finally understood:

In this architecture the DNN is used to transform the noise into phones.

In phonetics and linguistics, the word phone may refer to any speech sound or gesture considered as a physical event without regard to its place in the phonology of a language.

enter image description here

The last layer of the DNN is formed by all the possibles phones, having one output neuron per phone. The activation of these neurons is the probability of that the input noise corresponding to that phone.

The combination of these activations is the input of the Hidden Markov Model and establishes the senones of the HMM, which obtains a list of candidate texts by means of a dictionary.

The senones are the states of the HMM, in the following image, the senones would be x1 x2 and x3.

enter image description here

Please correct me if I've said something wrong, hope it helps!

$\endgroup$
1
  • 1
    $\begingroup$ @daviddavid, I added an answer which should explain a bit more clearly what a senone is. The whole senone/acoustic modelling thing is a bit messy in literature. $\endgroup$
    – Emiswelt
    Apr 24, 2018 at 15:34
2
$\begingroup$

In Speech Recognition, we often include some context about neighboring phones when modelling a certain phone. This means that our system not only knows phones for A, B and so on, but instead has a concept for E-then-A, O-then-B, X-then-A and so on.

These context-depended units are called senones in literature, which is of course a made-up word.

For speech recognition systems, these senones usually equal the HMM states of the acoustic model, which can be predicted by a neural network, if a DNN/HMM hybrid approach for acoustic modelling is used.

The term Senones was coined by the developers of the Janus speech recognition toolkit. It was then adopted by Dong Yu and Li Deng for their ASR book. That was in the time before NNs were used for acoustic modelling. Therefore the term is confusing.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.