What is the difference between and a Dense and a Sequential Layer in Keras?
They seem to be both just another layer in a neural network.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
In Keras, "dense" usually refers to a single layer, whereas "sequential" usually refers to an entire model, not just one layer. So I'm not sure the comparison between "Dense vs. Sequential" makes sense.
Sequential refers to the way you build models in Keras using the sequential api (
from keras.models import Sequential), where you build the neural network one layer at at time, in sequence: Input layer, hidden layer 1, hidden layer 2, etc...output layer. This is straightforward and intuitive, but puts limitations on the types of networks you can build.
Contrast this to the functional api (
from keras.models import Model), where you can build acyclic graphs, shared layers, etc....but where you have to specify a lot of the parameters yourself (e.g. how layers should be connected, which one is the input and which one is the output, etc...)
"Dense" refers to the types of neurons and connections used in that particular layer, and specifically to a standard fully connected layer, as opposed to an LSTM layer, a CNN layer (different types of neurons compared to dense), or a layer with Dropout (same neurons, but different connectivity compared to Dense).
Different types of layers can coexist in the same network, e.g. :
from keras.models import Sequential model = Sequential() model.add(LSTM(50, activation='relu', return_sequences=True, input_shape=(n_steps,n_features))) model.add(LSTM(50, activation='relu')) model.add(Dense(1)) model.compile(optimizer='adam', loss='mse')
Sequential is not a layer, it is a model. In sequential models, you stack up multiple same/or different layers where one's output goes into another ahead. This is the default structure with neural nets. Dense is a layer type (fully connected layer). There are others such as Convolutional, Pooling, LSTM etc.