I am an economist so my academic career has been spent on interpreting beta hat rather than optimising y hat. But I've become quite fascinated by neural networks so I wanted to get some things clarified.
So a 'regular' linear regression is an array of attributes with one hidden layer consisting of a single neuron. If we use the sigmoid activation function we get the logit model. Both of these I am very familiar with and I have used the first one for casual inference for many years (i.e., when can the beta's in y = BX + e be interpreted as causal effects on y).
My confusion arises when the hidden layer(s) have several nodes. I recall watching some TensorFlow lecture where it was said that the different nodes find different patterns. So in the MNIST dataset for example, if we wanted to detect the number nine, we know that it is composed of a circle with the bent line and a seven is a straight line combined with another straight line etc. So the different nodes in the hidden layers capture these patterns (please correct me if my interpretation here is incorrect).
So for image classification problems I kind of see the role different neurons play. But what about the case of regressions? Say I have a bunch of data about houses, what role do the different neurons play then? In the linear regression:
$Houseprice_{i} = a + \beta_1*size_{i} + \beta_2*No.Rooms_{i} + \beta_3*HasPool_{i} + e_{i}$
Each beta/weight tells me the effect each attribute has on the house price. But this requires just one hidden layer with a single neuron and an activation function that just maps to itself. If I add a bunch of hidden layers with several neurons I will most likely get a better prediction for the house price but how do I interpret the weights, number of neurons and number of hidden layers?
Any clarification would be greatly appreciated!
