I am trying to understand the following keras model:

in = keras.Input(shape = 76)

x = layers.Dense(80, activation='tanh')(in)
x = layers.Dense(70, activation='tanh')(x)
x = layers.Dense(60, activation='tanh')(x)
x = layers.Dense(50, activation='tanh')(x)
x = layers.Dense(40, activation='tanh')(x)
x = layers.Dense(30, activation='tanh')(x)
x = layers.Dense(20, activation='tanh')(x)
x = layers.Dense(10, activation='tanh')(x)

out = layers.Dense(1, activation='linear')(x)

model = keras.Model(in, out)

So in general, I would like to know what this model is doing. I also have two specific questions:

  1. Is there a mismatch between in the input tensor (shape=76) and the first layer's units (shape=80)? What effect does a mismatch have if any? How can 76 inputs go into 80 nodes/units?

  2. What is the purpose parentheses input at, for example "(in)" in the first Dense layer x = layers.Dense(80, activation='tanh')(in) or the "(x)" in x = layers.Dense(70, activation='tanh')(x) the second layer? I am not familiar with this type of notation in keras and I've looked everywhere online to for insight.

Thanks in advance!


1 Answer 1

  1. You are missing the point that the shapes are only one dimension, whereas the second one is implicit. I guess, this image showing both dimensions would be more helpful. As you can see, the shape of the input is $N \times 4$ and it is multiplied by $W_1$ with shape $4 \times 5$, so the shapes match as to multiply matrices you need only the "touching" dimensions to match.

Diagram showing a simple dense neural network with dimensions of the layers.

  1. You are using the functional API. layers.Dense(80, activation='tanh') returns an anonymous function that can be called. You could write it as well like
mylayer = layers.Dense(80, activation='tanh')

You can easily verify it by calling callable(layers.Dense(80, activation='tanh')), that confirms that the output is a function.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.