# What are the senones in a Deep Neural Network?

I am reading this paper: skype translator where they use CD-DNN-HMMs (Context dependent Deep neural Networks with Hidden Markov Models). I can understand the idea of the project and the architecture they've designed but I don't get what are the senones. I have been looking for a definition but I haven't found anything

—We propose a novel context-dependent (CD) model for large-vocabulary speech recognition (LVSR) that leverages recent advances in using deep belief networks for phone recognition. We describe a pre-trained deep neural network hidden Markov model (DNN-HMM) hybrid architecture that trains the DNN to produce a distribution over senones (tied triphone states) as its output

EDIT:

I've found this definition in this paper:

We propose to model subphonetic events with Markov states and treat the state in phonetic hidden Markov models as our basic subphonetic unit -- senone. A word model is a concatenation of state-dependent senones and senones can be shared across different word models.

I guess they are used in the Hidden Markov Model part of the architecture in the first paper. Are they the states of the HMM? The outputs of the DNN?

• A senone is a speech recognition term. Is that the definition you're looking for, or clarification of how they're modeled in that paper? – Sean Easter Dec 2 '15 at 13:15
• More like the application of then to the DNN-HMM. They are the states in the HMM but also de outputs of the DNN ? – davidivad Dec 2 '15 at 16:50
• This article describes senones in some detail ... cmusphinx.sourceforge.net/wiki/tutorialconcepts – DJohnson Dec 31 '16 at 20:43

This is what I've finally understood:

In this architecture the DNN is used to transform the noise into phones.

In phonetics and linguistics, the word phone may refer to any speech sound or gesture considered as a physical event without regard to its place in the phonology of a language.

The last layer of the DNN is formed by all the possibles phones, having one output neuron per phone. The activation of these neurons is the probability of that the input noise corresponding to that phone.

The combination of these activations is the input of the Hidden Markov Model and establishes the senones of the HMM, which obtains a list of candidate texts by means of a dictionary.

The senones are the states of the HMM, in the following image, the senones would be x1 x2 and x3.

Please correct me if I've said something wrong, hope it helps!

• @daviddavid, I added an answer which should explain a bit more clearly what a senone is. The whole senone/acoustic modelling thing is a bit messy in literature. – Emiswelt Apr 24 '18 at 15:34

In Speech Recognition, we often include some context about neighboring phones when modelling a certain phone. This means that our system not only knows phones for A, B and so on, but instead has a concept for E-then-A, O-then-B, X-then-A and so on.

These context-depended units are called senones in literature, which is of course a made-up word.

For speech recognition systems, these senones usually equal the HMM states of the acoustic model, which can be predicted by a neural network, if a DNN/HMM hybrid approach for acoustic modelling is used.

The term Senones was coined by the developers of the Janus speech recognition toolkit. It was then adopted by Dong Yu and Li Deng for their ASR book. That was in the time before NNs were used for acoustic modelling. Therefore the term is confusing.