# Common activation function in fully connected layer

I'm a newbie in deep learning. As I have known, each neuron has a gain/weight and an offset/bias with an activation function (e.g. sigmoid, tanh, ReLU, identity and etc).

In the convolution layer in a Convolution-Neural-Networks mentioned that it usually concatenate with ReLU activation function, but what happened in the fully connected layer?

What is the most common activation function in a fully connected layer in a deep CNN?

• Fully connected input layer (flatten)━takes the output of the previous layers, “flattens” them and turns them into a single vector that can be an input for the next stage.
• The first fully connected layer━takes the inputs from the feature analysis and applies weights to predict the correct label.
• Fully connected output layer━gives the final probabilities for each label.
• Most people would call y=x the "identity" activation. – Sycorax Aug 27 '19 at 12:58
• That's right, I'll update it. – Benyamin Jafari Aug 27 '19 at 13:43

ReLUs and similar functions are popular because they speed up network training, as they only have a flat gradient on one side instead of two, as is the case with $$\tanh$$ and sigmoid activations.