I'm learning how to apply neural networks. In the example below, I use a linear activation function. Does this mean that the model becomes multivariate regression or is it an enhanced regression that covers discontinuous function due to layer and thresholds that control whether neurons fire up to the next ones
Per the documentation, the call
nn <- neuralnet(consumption ~ ., data=scaled, hidden=c(3), algorithm = "rprop+", linear.output=TRUE, threshold=0.001)
actually only uses a linear activation function for the final output node, the one labeled
consumption in your figure. The interior hidden nodes use the default value of
act.fct = "logistic", a logistic (also called sigmoid) activation function. Thus your model is capable of learning nonlinear functions.
There is no thresholding going on in your code, contrary to what you seem to think is happening. That would correspond to some different choice of activation function (perhaps a ReLU, among other options) that is zero if the input is below some threshold.
If you want to really use a linear activation function throughout, you can pass a linear function to
act.fct. But then you'll just be learning a linear regression, as described by Dave's answer, with an unusual optimization scheme.
"Multivariate regression" means that your response variable is a vector. "Multiple linear regression" is the term you mean, and using linear activation functions turns your neural network into a multiple linear regression. Your layers are linear transformations over and over, and the result of composing linear transformations is another linear transformation.
The power of a neural network is in its ability to figure out all kinds of nonlinear decision boundaries. Using ReLU activation functions, you can get two neurons to give a decision boundary in the shape of $\vert x \vert$. Using more neurons, you can approximate $x^2$ or $x^7 - 13x^4+x^2$ (or plenty else). Without the nonlinear activation function, you're only doing linear decision boundaries.
What you're doing with a threshold for neurons firing at all confuses me and, from the comments, others. Could you please clarify that?