Questions tagged [speech-recognition]

Automatic speech recognition (ASR) aims to identify words and phrases in spoken language and convert them to a machine-readable format.

Filter by
Sorted by
Tagged with
1 vote
1 answer
38 views

Comparing data with opposite distributions

What test can I use to compare whether the difference between the following sets of data are significant. I know, I can just look at them, but I'd like it to be a bit more scientific than that. The ...
Mariano Mazzeo's user avatar
0 votes
0 answers
22 views

Proven approaches for labeling audio data for training a speech-to-text model

To train a deep learning model, for converting speech to text, we need labeled data. How should this data be arranged? I can think of several approaches, and I don't know which approach has already ...
google dev's user avatar
0 votes
0 answers
24 views

How does LSTM perform inference on untrimmed video or audio?

I am new to the concept of RNN and LSTM. The concept of LSTM gave me an impression that it performs inference step by step. If the input data is a video, it first consumes Frame(t-1), and then ...
Jim Wang's user avatar
  • 113
0 votes
0 answers
22 views

How to incorporate real-valued multi-observations into Hidden Markov Model?

I want to perform Voice Activity Detection (VAD) application which decides whether there exists human voice in the audio signal or not. I want to train a HMM using Baum-Welch algorithm. The states ...
sdkmlcngz's user avatar
2 votes
1 answer
258 views

Approaches for semi-supervised fine-tuning after self-supervised pre-training

My understanding is that self-supervised learning approaches approximately work like the following (I have Wav2Vec 2 in my mind here, used in speech recognition, but NLP transformer models are similar)...
phipsgabler's user avatar
1 vote
0 answers
29 views

Hidden Markov models in Speech Recognition

My first question here. So I am trying to build a sign language translator(from signs to text) and noticed that the problem itself is quite similar to speech recognition, so I started to research ...
Raiymbek Akshulakov's user avatar
1 vote
0 answers
1k views

Why almost all neural speech processing involves Mel Spectrograms?

What are the reasons behind almost all speech processing whether it be generative or recognition heavily based on Mel Spectrograms? In a conversation with a signal processing expert I was asked why ...
Rijul Gupta's user avatar
2 votes
0 answers
73 views

How to decode an output from a phoneme recognition model?

I created a phoneme recognition model (based on a pre-embedding with Wav2vec, and some layers on top of it) that takes as input an audio signal, and outputs a softmax matrix of size $N \times C$, ...
Hugo Laurençon's user avatar
0 votes
0 answers
44 views

CTC Speech Recognition Model giving absurd results on actual recording

I have trained a speech recognition model which uses CTCLoss and is inspired from https://www.assemblyai.com/blog/end-to-end-speech-recognition-pytorch I trained it on the Librispeech Dataset (train-...
Divyaanand Sinha's user avatar
2 votes
0 answers
91 views

Confusion about the derivative in CTC

I was going through the original CTC paper by Graves et al, I am still not getting how after taking the derivative of equation 14 we get equation 15 as shown below I understand the part that we are ...
Divyaanand Sinha's user avatar
0 votes
1 answer
85 views

Speech recognition (SVM) different signal lengths

I am developing a small project on speech recognition, the idea is to classify sound sources by Support Vector Machines. My dataset consists on 45 signals, however, they all have different lengths, ...
Berna Gomez's user avatar
0 votes
1 answer
230 views

Why does one need Google's WaveNet model to generate audio if it already takes audio as input?

I've spent a lot of time trying to understand the Google's WaveNet work (also used in their DeepVoice model), but still confused about some very basic aspects. I'm referring to this Tensorflow ...
Joe Black's user avatar
  • 319
2 votes
1 answer
451 views

Mismatching dimensions of input/output in the WaveNet model for text-to-speech generation?

I have been trying to understand the model of how speech generation works, particularly in WaveNet model by Google. I was referring to the original WaveNet paper and this implementation: I find the ...
Joe Black's user avatar
  • 319
1 vote
2 answers
469 views

How are text-to-speech systems' spectrogram frames aligned for loss calculation?

A key aspect of how text-to-speech (TTS) machine-learning works is very unclear to me even after reading the Tacotron-2 paper and the Google AI blog. https://ai.googleblog.com/2017/12/tacotron-2-...
Joe Black's user avatar
  • 319
0 votes
1 answer
219 views

Aggregating LSTM subsequence output into full sequence

I have an n->n seq2seq LSTM that takes a sequence of length n and produces a sequence of length ...
Harry Stuart's user avatar
1 vote
1 answer
473 views

Speaker Recognition ML tasks are supervised or unsupervised?

Given the scenario: We have a speech recording from an unknown person. We have a speech recording from a known person. We have a large database of speech recordings from different persons. We would ...
Hordon's user avatar
  • 111
1 vote
0 answers
177 views

Question about Connectionist Temporal Classification (CTC) gradient

I have read the original CTC paper by Graves et al, but am confused about equation 16, for which the authors derive the gradient of the negative log likelihood objective with respect to the inputs of ...
minch's user avatar
  • 161
0 votes
1 answer
55 views

Interpolating speech from different speakers in trigger word detection

Trigger word detection aims to essentially identify the timestamp of a "trigger word" occurring in a chunk of audio. The approaches I have seen online, in particular, following along with Andrew Ng's ...
Harry Stuart's user avatar
0 votes
2 answers
247 views

Does speech recognition model training require transcript timestamps

I don't quite understand how a recurrent neural network or LSTM is trained for automatic speech transcription. Say I have n audio files of speech, each with an ...
Harry Stuart's user avatar
2 votes
1 answer
115 views

Method for detecting previously unseen class

Is there any common practice for detecting a new class, or data associated with an previously unseen event? I'm doing some research into speech recognition, and I'm trying to detect when a speech ...
Cerin's user avatar
  • 654
2 votes
1 answer
941 views

Number of parameters of tacotron, deep voice, wavenet?

I have recently started to explore speech synthesis, and started reading some paper. I have implemented a dummy text to speech synthesis model too, it has around 92 million parameters. Even though, ...
Zabir Al Nazi's user avatar
0 votes
1 answer
147 views

k-means clustering issue voice data

I'm getting an issue in my k-means I don't know if it my data-set or what anything else. Why i got thia flowing point in the right side of the image? ...
abdoulsn's user avatar
  • 115
2 votes
1 answer
254 views

When use CTC-loss for speech recognition?

I'm trying to understand and implement CTC-loss for speech recognition (here on SO). I'll like to have more information about the use cases of this technique. From what i understood, it is more ...
Baptiste Pouthier's user avatar
1 vote
1 answer
3k views

How do you evaluate/test accuracy of Text-to-Speech (TTS) models?

As the title implies... For instance, for Machine Translation, we have BLEU. For categorization, we have categorical crossentropy, for binary categorization, we have binary crossentropy, etc. etc. ...
gust's user avatar
  • 113
2 votes
1 answer
185 views

Streaming audio to neural network

I am trying to create a neural network that performs speaker recognition. I would like to be able to serve it such that it takes streaming audio - i.e. I want to perform partial recognition on 100ms ...
Harry Stuart's user avatar
1 vote
1 answer
33 views

Is there a keyword recognition system without learning the Phoneme

As I understand in speech processing and machine learning area "keyword recognition" (also termed as keyword spotting) is a important part. In "keyword recognition" sytem it is desired to learn a ...
Creator's user avatar
  • 229
0 votes
1 answer
192 views

Deciding length of units in sound recognition for training HMMs

I am working on creating a method to detect changes from one song to another. Namely, I hope to use a Hidden Markov Model (HMM) in order to model a part of a song and check to see if it accurately ...
somil's user avatar
  • 103
1 vote
1 answer
274 views

Are e2e DL systems better than DNN-HMM models in speech recognition?

End-to-end deep learning systems for automatic speech recognition (ASR) have been around for a while now since Deep Speech (2014), but I noticed that DNN-HMM based methods are still performing well ...
Ahmed Maher's user avatar
2 votes
1 answer
106 views

How to use GMMs for acoustic signal classification?

There are a number of applications of the Gaussian Mixture Model (GMMs) to acoustics/audio data for the purposes of classification; ex paper1 and ex paper2. GMMs ...
Vass's user avatar
  • 1,665
1 vote
0 answers
265 views

How to create dataset for speech recognition using librosa [closed]

I loaded the audio using librosa and extracted mfcc feature of the audio. I now have array of shape (20,N). How do I feed this as input to LSTM to predict?
aneesh cool's user avatar
1 vote
1 answer
2k views

WaveNet Global and local conditioning

WaveNet is a deep learning framework able to generate raw audio signal from a sequence like text sequence. https://arxiv.org/abs/1609.03499 It is also possible to "imitate" in a way the voice of the ...
Shiro's user avatar
  • 123
1 vote
1 answer
195 views

Speech recognition - hangover scheme - voice activity detection

I am doing a voice activity detection challenge, and I am asked to add a hangover scheme to the model. I read about hangover schemes in different papers but I couldn't find a definition for this. What ...
Robin's user avatar
  • 233
2 votes
2 answers
2k views

What is low rank linear layer in neural networks?

Going through the paper Convolutional Neural Network for Small-footprint Keyword Spotting. In the paper, authors have used low rank linear layer after convolution ...
Arunesh Singh's user avatar
2 votes
1 answer
52 views

Verifying Time Warp

Time warp has been widely assumed in domain of speech processing. If $Xw(t)$ represents a time warped version of $X(t)$, then $Xw(t) = X(t-w(t))$ where $w(t)$ is an arbitrary function with a banded ...
Mike Zadeh's user avatar
1 vote
0 answers
256 views

How does CTC work in Speech Recognition?

I have already read the 2006 Paper about CTC by Graves but I still don't understand it fully. I am searching for a simple but still detailed explanation of how "Connectionist Temporal Classification" (...
Niklas Donges's user avatar
1 vote
1 answer
254 views

RNN(LSTM) model fails to classify new speaker voice

I'm fairly new to ML and at the moment I'm trying to develop a model that can classify spoken digits (0-9) by extracting mfcc features from audio files. My data ...
Moras's user avatar
  • 13
0 votes
2 answers
539 views

Transfer learning for audio

I know that when working with images, what people normally do is download a big model trained with huge data and freeze most of the layers except the lasts ones to train them with their own data. I'm ...
Isaac's user avatar
  • 115
3 votes
0 answers
1k views

Validation loss is less than training loss by 5 units. How this result is interpreted?

Iam training a Keras model for end-to-end speech recognition. I have my own dataset of speech containing about 400 wave files. Text transcriptions is also given as input. Model summary is: ...
ml_user0993's user avatar
0 votes
1 answer
104 views

Adapt speech recognition for Shakespeare english

We need to be able to search the works of Shakespeare by voice. The way I see it, the goal is if I quote into the microphone: "Yet but three come one more. Two of both kinds make up four. Ere ...
AwokeKnowing's user avatar
7 votes
1 answer
4k views

Turkish speech recognition (speech->text) in Google Speech API? [closed]

Google's Speech API has audio speech to text capabilities in multiple languages. It supports Turkish too. That language is very interesting, it's so called agglutinative: you stick word parts one ...
Aksakal's user avatar
  • 60.1k
3 votes
1 answer
1k views

Word Error Rate over Data Set

In speech to text, one common metric is the word error rate (WER). WER is the word-level Levenshtein distance, which is the minimum number of substitutions ($S$), deletions ($D$), and insertions ($...
user18764's user avatar
  • 161
1 vote
1 answer
1k views

How to normalize text when computing the word error rate of a speech recognition system?

I am looking for a library, script or program that can normalize the transcribed and gold texts when computing the word error rate (WER) of an automated speech recognition system. For example, if: ...
Franck Dernoncourt's user avatar
0 votes
1 answer
85 views

Spot word in spoken sentence : advice needed

I have to make a live speech recognition program that can spot specific words in a spoken sentence. For now I have to recognize the words "yes" and "no". I already trained Google's model and it ...
Omar Aflak's user avatar
0 votes
1 answer
122 views

How to prevent the creation of redundant mixtures while training a GMM?

I'm currently trying to train a GMM(UBM) with 1024 Gaussian mixtures for speaker verification. However, after training the GMM, it appears that some mixtures are useless/redundant. (little to no ...
whkang's user avatar
  • 179
1 vote
0 answers
600 views

Keyword Spotting: How to train a model with general speech corpus?

I am trying to find a correct way to train a DNN based keyword spotting (Deep KWS) with general speech corpus (VS data) described in this paper (Chen, Guoguo, Carolina Parada, and Georg Heigold. "...
Lion Lai's user avatar
  • 135
2 votes
1 answer
142 views

MFCCs and MoG-HMMs for speech recognition

BACKGROUND MFCCs are coefficients which represent the most important parts of speech, and about 12 of them are used to model a one 512 points long frame (of speech). Along with them you would use ...
Desperado's user avatar
  • 131
0 votes
1 answer
56 views

Why is phase reconstruction considered hard

I am studying deep learning models for single channel speech separation. I come across several recent methods: Permutation Invariant Training Deep Clustering Deep Attractor Network All of these ...
Kh40tiK's user avatar
  • 149
2 votes
2 answers
365 views

Why has deep learning only shown decent results in the fields of computer vision and speech recognition? [closed]

We all know about the success of ImageNet, AlphaGo etc which used deep neural networks in computer vision, or the use of RNNs in Google Translate. But why are we not seeing similar advances in other ...
user3055163's user avatar
3 votes
0 answers
234 views

Program to evaluate the output of a speech recognition system

I am looking for a library, script or program that can evaluate the output of a speech recognition system. The output of the speech recognition system is a simple text file, and I have the gold output ...
Franck Dernoncourt's user avatar
1 vote
1 answer
122 views

how we input speech signal waveforms in two deep learning algorithms?

I am working with deep learning algorithms like CNN and RNN.I always wonder what is the best way to input wave form type data in to the deep learning algo. I know there are methods like wavelet or mel ...
Shamane Siriwardhana's user avatar