I'm a newbie in deep learning. As I have known, each neuron has a gain/weight and an offset/bias with an activation function (e.g.
identity and etc).
In the convolution layer in a Convolution-Neural-Networks mentioned that it usually concatenate with
ReLU activation function, but what happened in the fully connected layer?
What is the most common activation function in a fully connected layer in a deep
- Fully connected input layer (flatten)━takes the output of the previous layers, “flattens” them and turns them into a single vector that can be an input for the next stage.
- The first fully connected layer━takes the inputs from the feature analysis and applies weights to predict the correct label.
- Fully connected output layer━gives the final probabilities for each label.