I was learning that seq2seq models (from the deeplearning.ai course) try to maximize:

$$ \max_{y} P_{\theta}(y_1 \dots y_{T'} \mid x_1 \dots x_T ) $$

I learned that one way they do it is via beam search. However, what I don't understand is where SGD comes in the picture. I will explain the way I think it comes in the picture and the true objective I think we are trying to optimize. I think we are actually trying to optimize:

$$ \max_{\theta,y} P_{\theta}(y_1 \dots y_{T'} \mid x_1 \dots x_T ) = \max_{\theta,y} J(\theta,y) $$

we want to optimize the most likely sequence but we also want to improve the RNN parameters (or at least that's what I would have thought, specially since in normal supervised learning thats what we do).

So according to that I believe this is the way it would be done:

  • at iteration $t$ we have model $\theta^{(t)}$
  • we get it's most likely sequence with beam search $\hat y = y_1 \dots y_{T'} := BeamSearch(x_1...x_T,\theta^{(t)})$
  • the using that most likely sequence we do SGD on the parameters according to the (normalized log likelihood) $$\theta^{(t+1)} := \theta^{(t)} - \eta_t J(\theta^{(t)},\hat y)$$ which basically makes the current most likely sequence even more likely with SGD

  • repeat

I think this is probably wrong because it would seem this algorithm reinforces a specific sentence. If the first guess wasn't right I'd assume it would be some sort of horrible confirmation bias algorithm...but I can't see how SGD comes in the picture according to the descriptions I've read in Coursera's deeplearning.ai!


A sequence to sequence model produces a probability distribution over sequences $Y$ conditioned on another sequence $X$.

At test time, you may want to find the mode of that distribution: $\max_Y P(Y|X;\theta)$ -- the sequence which is most likely. For example if you have a translation model, you want to find the translation which has the highest probability under the model. This maximization must be done with beam search.

At training time, you already have $X,Y$ pairs from the data and you are simply trying to maximize the (log) likelihood of the model: $\max_\theta E[\log P(Y|X;\theta)]$. This is a completely different task from test time, and doesn't require beam search, and is done with SGD.

  • $\begingroup$ So beam search is only during test time? $\endgroup$ Jun 1 '19 at 18:17
  • 1
    $\begingroup$ @Pinocchio in this context -- yes $\endgroup$
    – shimao
    Jun 1 '19 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.