# Questions tagged [forward-backward]

The forward-backward algorithm is an inference algorithm for Hidden Markov Models (HMMs). DO NOT use this tag for forward or backward regression; use stepwise-regression instead.

35 questions
Filter by
Sorted by
Tagged with
21 views

### Time Duration Modelling for Hidden Markov Models

As we know, one of the major weakness of conventional HMMs is related to the modelling of state duration, as the probability of state occupancy decreases exponentially with time. There are multiple ...
40 views

### Block sampling hidden state using forward algorithm only

In a hidden Markov model, I can't get my mind around why I can't sample the full hidden state $\vec x$ using only a forward sampling algorithm. Let $\vec y$ be the observed data and $\theta$ the model ...
1 vote
45 views

### Scaling factor in forward-backward algorithm

I am studying forward-backward algorithm following the wikipedia page. I have little background in statistics and have managed to understand (hopefully) most part of the algorithm. However the scaling ...
1 vote
100 views

### Select polynomial order for continuous variables in mixed model step-wise backward selection

I am working on some data that I would like to analyze through a generalized linear mixed model regression and a stepwise backward selection of variables directly on that model. I use the GLMERSelect ...
12 views

### How to efficiently caclulate probability of a state in HMM when a random shuffle operation happens on emitted observations

The problem setup is as follows(it's from a book and may not be tied to reality): Suppose we have some speakers s_1, s_2..s_k seated around a table speaking at ...
115 views

### Computing confidence intervals backward elimination with bootstrapping

I do backward elimination, by iteratively removing the biggest p-values until the biggest p-value is < 0.157. Then, I have a model, which confidence intervals displayed are not wide enough: "...
187 views

### Computing the partition function using the forward-backward algorithm for linear chain CRFs

I'm trying to implement the forward-backward algorithm for a Linear Chain Conditional Random Field, as to compute the marginal distribution over labels for each time step in a sequence. I'm following ...
959 views

### Are there any way of removing impact of a certain data from a trained model (about "right to forget")

I was reading about wearable technologies (Recent Advances in Wearable Sensing Technologies). They briefly talk about Right to forget and a question came to my mind. Suppose that we trained a deep ...
822 views

### What do they mean by "batchnormalization allows to initialization of weights less carefully?"

In Towards Data Science - Manish Chablani - Batch Normalization, it is stated that: Makes weights easier to initialize — Weight initialization can be difficult, and it’s even more difficult when ...
127 views

### Multicollinearity for Logistic Regression and Neural Network

I am looking to fit Logistic Regression (LR) and Neural Networks (NN) models in order to predict if there will be avalanches during a day (0 or 1 dependant variable) based on meteorological variables (...
289 views

### Why use dropout in feedforward?

Maybe I am just confused by what is the point of using dropout in the feed-forward? Wouldn't be better to forward the input with the whole network and then use the dropout only in the back-prop to ...
218 views

1 vote
233 views

### Using Forward Backward algorithm to find posterior probability of all possible states

I understand that Viterbi finds the most probable sequence of states. However, I want the probability of all possible sequences of states. I understand that FB algorithm can be used to find the ...
1 vote
431 views

### Hidden Markov Model: How to define the state observation matrix B for continuous (Normal) observations?

I am having a hard time understanding how to use the observation matrix B for continuous HMM assuming the observation of each hidden state Normal. So far I defined the matrix B as an Nx2 matrix where ...
865 views

### Why does forward selection only take $O(p^2)$ calls to the learning algorithm?

In http://cs229.stanford.edu/notes/cs229-notes-all/cs229-notes5.pdf pg 5, it states that forward search takes $O(p^2)$ (note the notes uses $n$ instead of $p$ for the number if independent variables). ...
358 views

### The difference between forward algorithm used in CRF and the variable elimination?

I found that in the forward algorithm used in the CRF(and perhaps also in the HMM) the mechanism applied is almost the same as that in the variable elimination(VE) except that the emission ...
1 vote
2k views

### Example of manual implementation of baum-welch algorithm in R

Is there any code out there that implements the baum-welch algorithm for a very basic problem? It would be very helpful to actually see the algorithm in action to better understand how it works. I ...
1k views

### Viterbi and forward-backward algorithm in HMM

I am learning HMM recently and got confused with the training problem (training model parameters and hidden state given outcome sequence). As far as I know, both Viterbi learning and Baum-Welch (...
1 vote
572 views

### HMM - Difference between forward and backward case

In the HMM formulation where z is hidden state and x is observed In the forward case, I see it represented as such: $\alpha_{k}(z_{k})=P(z_{k},x_{1:k})=\sum_{z_{k-1}}P(z_{k},z_{k-1},x_{1:k})$ but ...
518 views

### Trouble understand HMM Forward Algorithm

Referencing: https://en.wikipedia.org/wiki/Forward_algorithm I understand most of the expansion of the forward algorithm, but the very first step is confusing to me: Why does the joint probability ...
781 views

### How to sample an unobserved Markov process using the forward-backward algorithm?

The setup Let $X = (x_1, \ldots, x_T)$ denote a state variable that follows a Markov process, where $x_t \in S$. The transition distribution is denoted by \begin{equation} p(x_{t}|x_{t-1}) . \end{...
755 views

### RNNs: backprop loss from just the last time step or every single one?

Consider a simple task that predict the next alphabets based on previous ones using RNNs. That is, during model inference, we would like the model to output y1_hat (...
1 vote
759 views

### HMM forward algorithm in R [closed]

maybe a very basic hmm implementation question. Would there be a way to determine the most 'likely' of two different HMMs for a specific sequence? I was thinking about using the forward algorithm ...
215 views

### Combining the forward and backward algorithms in HMMs

Suppose that I have a Hidden Markov Model and want to estimate the probability of a sequence of observations. Furthermore, not only do I wish to know the total probability of the sequence, but I want ...
44k views

### Forward-backward model selection: What is the starting model?

I am trying to understand the logic behind forward-backward selection (even though I know that there are better methods for model selection). In forward model selection, the selection process is ...
2k views

### Forward-backward algorithm for HMM

I am currently studying this paper In which i am having some problems understanding the purpose of the forward-backwards algorithm. First of all why even have both forward and backwards? It seems ...
1 vote
1k views

### Implementation of Forward Backward Algorithm

While studying about Forward Backward Algorithm, I came across the question below. I was unable to solve the question after trying for a lot of time. Consider a two-bit register. The register has ...
1 vote
2k views

### Faster alternative of forward/backward model selection for big datasets

I want to perform a forward/backward selection to build a predictive model. My data set is very large though and if I include all variables in the selection process it is way too slow. Therefore I am ...
257 views

### Significance of covariates in mixed model analysis

I have a question about the significance of adding covariates in a mixed model that I can't explain, or find an answer for online. I hope that by posting my question here, someone may know a good ...
1 vote
61 views

### Implementing forward-backward algorithm: confusion around presence of start and stop states in transition matrix

I'm attempting to implement the forward-backward algorithm. I have my observations and I am attempting to estimate $A$ and $B$. Looking at the algorithm, one thing I'm having trouble figuring out, is ...
1k views

### Logistic regression variable selection methods

I'm having trouble to understand Backward elimination in Logistic Regression model. I was looking at this example of Agresti, Categorical Data Analysis, to see how Backward elimination works. What I ... 1 vote
510 views

### Forward-Backward Algorithm explanations

I am studying the Forward-Backward Algorithm from HMMs and the forward-backward algorithm by Ramesh Sridharan I don't understand how to get $(1,3,4)^{T}$ in the example below. Could someone help to ...
I am not very clear on what is the proper way to train an RNN. Suppose we are using a vanilla RNN and are given some categorical sequence $x$ of length $T$: $$x= [ x_1,\ldots,x_T]$$ To fit the ...