I came across this answer which is specific to Keras. But my question is at concept level. I am getting confused, How can RNN handle variable size inputs? The answer here says RNN can handle variable sequence inputs.
My question is, How exactly the weights matrix W (see my figure) work with different length inputs?
Let us suppose we want to do a sentiment classification.
Training batches of different length : Which I think means all training examples in a batch must be of same length but different batches can have variable length training examples.
Now, suppose length of training examples in
- batch-1 = 10,
- batch-2 = 15.
- batch-3 = 18 ... so on
Now while training batch-1 , In the unrolled RNN there would be 10 columns in the below figure to handle input of sequence 10. By columns I mean different timestamps in RNN. The shared weights W, U,V dimensions must be in sync with sequence length. What I mean by this is, the shared weights to handle sequence of length 10 would be something like w1,w2,..w10 (horizontal connection that connects columns.) I understand till this part.
Now, while processing batch-2 which is of sequence length 15, The unrolled RNN must take sequence of length 15. That means there are 15 columns in the below figure and corresponding weights will be like w1,w2,..w15.
The number of weights W changed from 10(w1,w2,...w10) to 15(w1,w2,..w15) as we go from batch-1 to batch-2.
The W dimension changes. Things don't add up here for me.
What happens to shared weights? How are they learnt?
