Is the cost function of a neural network with only linear activation functions a convex function with respect to its parameters? If it is, how to prove it?
I'm opening up the statement made in the comments.
A neural network with linear activation and many layers is equivalent to a neural network with one layer. For example:
So, in the end, your equation is $Wx+b=y$, i.e. just a linear regression. The cost function is already convex, because it is norm-squared of an affine function. Also, more strongly, the problem is a convex problem.