Mathematical structure of SimpleRNN in keras

Question

Two types of RNN can be used:

Type1: The output is being used as state

h(t) = g(W₁.x(t) + W₂.h(t-1) + b₁)

Type2: There is a state in addition to the output

a(t) = g(W₁.x(t) + W₂.a(t-1) + b₁)

h(t) = g(W₃.a(t) + b₂)

Which of the above mathematical structure is implemented by SimpleRNN in keras? I tried looking at the documentation, but its unclear.

Edit: The source of keras.layers.SimpleRNN suggests RNN of type 1. See the call method of SimpleRNNCell at https://github.com/keras-team/keras/blob/master/keras/layers/recurrent.py

Answer 1

The documentation already linked says it all:

"""Fully-connected RNN where the output is to be fed back to input.

So simply said, the output of your previous step is used as the recurrent input for the next step. Therefore you are dealing with the first type you described.

Answer 2

The documentation for the SimpleRnn defines the argument:

return_state: Boolean. Whether to return the last state in addition to the output.

Which would imply that states and outputs are regarded separately.