# Neural Networks Perceptrons and MLPs

While studying as a newbie about Neural Networks I started as everyone from the basics (perceptrons, MLPs) then how backpropagation works before dive in to harder deep learning concepts.

Now, I am trying to solve exercices (theoretical and practical) in order to practice and I have two questions.

1) I read that the Perceptron (single neuron) relates to a logistic regression classifier in particular, a (binary) logistic regression classifier is the same as a (single neuron) Perceptron with a sigmoid activation function. Is it true? How can I prove it? any link/example with examples etc.

2) A Multi-layer NN without activation function is equivalent to applying a linear transformation to the input data. Can someome explain me why this is true? an example?

In a binary logistic regression classifier, you have two scalar outputs: $$S(x) = [S(x)_0,\hspace{0.2cm}S(x)_1]$$, where $$S$$ is the softmax function. The prediction would then be the class0 if $$S(x)_0 > S(x)_1$$, and class1 if $$S(x)_0 < S(x)_1$$. Since we know that $$S(x)_0 + S(x)_1 = 1$$ (property of softmax), then we can say that the prediction would be class0 if $$S(x)_0 > 0.5$$ and class1 if $$S(x)_0 < 0.5$$ (you can check that this is equivalent to the previous statement). Already you can see that we do not actually need $$S(X)_1$$ to make a prediction, but we can make a prediction having only the value of $$S(X)_0$$. Now let's see how we can calculate the value of $$S(X)_0$$ using sigmoid: $$S(x)_0 = \dfrac{e^{x_0}}{e^{x_0}+e^{x_1}}= \dfrac{1}{1+\frac{e^{x_1}}{e^{x_0}}}= \dfrac{1}{1+e^{-(x_0-x_1)}}=\sigma(z)\quad\text{(where z=x_0-x_1)}$$ $$z=x_0-x_1=(w_0-w_1)a+(b_0-b_1)=w'a+b'$$
So all you need to do is use a single node with the weight vector $$w'=w_0-w_1$$ and the bias $$b'=b_0-b_1$$ to calculate $$z$$ (where $$w_0$$ and $$w_1$$ are the weights corresponding to the two nodes of the binary logistic regression model), and then proceed to take the sigmoid of $$z$$.
2- Suppose a 3 layer neural network without activation functions: $$A_1 = W_1X\\ A_2 = W_2A_1\\ A_3 = W_3A_2$$ $$X$$ is our input matrix where each column contains an input sample, $$W_i$$ is the weight matrix, and $$A_i$$ is the output at the $$i$$'th layer. Then we can simply write $$A_3$$ as: $$A_3 = W_{3}(A_2) = W_{3}(W_{2}A_1)= W_{3}(W_{2}(W_1X))=(W_{3}W_{2}W_1)X= W'X$$ The output ($$W'X$$) is just a linear transformation ($$W'$$) applied to your inputs ($$X$$).