1
$\begingroup$

I want to implement a Recurrent Neural Newtork (RNN) and use it for a classification task. I can handle a Feed Forward Neural Network and I followed this blog tutorial to learn more about the implementation of a RNN. My main concern is about the dimension of $x_t$ because the tutorial encodes words in one-hot vectors, which I don't need. Consider the computation:

$$h_t = \sigma(Ux_t + Wh_{t-1})$$

Here $\sigma$ is some activation function and $h_t$ refers to the "hidden" state in $t$. Given that one input example $x$ would be a sequence of real valued numbers, say 784 numbers if I use the MNIST dataset for testing, and such a sequence belongs to one class, what would be the dimension of $x_t$? I suspect it is just a scalar (1 by 1, esentially one number of the sequence) but I would like to confirm with someone who has experience with RNNs. The dimension of matrix $U$ would be HiddenLayerSize by DimOfX_t and matrix $W$ would be of dimension HiddenLayerSize by HiddenLayerSize, I believe. What would need to adapted for a regression task? Many thx for any help in advance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.