I don't think that anything about a neural network is easier to understand  by using the "independent variable" and "dependent variable" terminology. It's easier to think about neural networks in terms of inputs and outputs. A neural network takes something (a sequence of integer indices, a vector, an image, several vectors concatenated into a matrix, a graph) and returns something else (a probability vector, an arbitrary vector, another image). 

And of course there are some neural networks that can take multiple, heterogenous inputs and return one or more outputs (which may, likewise, be heterogenous).

This description is incredibly abstract and general. That's kind of the point: neural network researchers have generalized beyond what is possible with linear regression.

What a "feature" is depends on context. Some recent successes in neural networks are entirely featureless, in the sense that they take some raw input, such as an entire *image*, as the input. This is in contrast to other image-processing tasks which take an image, extract features, and then pass the features to some downstream task.

Other applications of neural networks are exactly like linear regression with extra bits added on: matrix input, scalar output. The only subtlety is the hidden layer nonlinearity.

Hyperparameters are not independent variables. Let's return to the regression context. One hyperparameter of a ridge regression is the penalty on the $L^2$ norm of the coefficients. This is not an independent variable because it is not an attribute of one of the samples in your data collection; instead, it's a researcher-chosen value which controls the length of the norm of the coefficients.

Likewise, neural network are not independent variables. Hyperparameters, including the $L^2$ penalty and learning rate, don't describe anything about your data set; it's just a direct consequence of using an iterative optimization procedure.

This thread may also be useful. https://stats.stackexchange.com/questions/362425/what-is-an-artificial-neural-network/362427#362427