Yes you can interpret them as but, as you said, in most cases these variables have no real world interpretation because they are arbitrary linear combinations of the actual input variables.
However, in some networks, these variables aren't completely meaningless and can be interpreted. One such example are Convolutional Neural Networks (CNNs), which somewhat preserve the spatial information of their input due to their sparse connectivity.
An example where this idea is used is to produce the so called Class Activation Maps from a CNN. These essentially tell us "where a CNN is looking when it makes a prediction".
I will briefly explain the main concept, however it's best to read the actual paper which has
Imagine a simple CNN with four layers:
conv1 -> conv2 -> conv3 -> fc
The input of
conv1 is an image that may have $3$ channels if it isn't grayscale. The output of the fully connected layer
fc a the softmax output that classifies that image. Now, the output of the
conv3 layer is another image with an arbitrary amount of channels (let's say $100$). Due to the fact that
fc requires a 1D input, the output of
conv3 is usually flattened before being fed to
fc, so the actual CNN would look like this:
conv1 -> conv2 -> conv3 -> flatten -> fc
Let's take a look at the shapes of the tensors as they pass through the network. Let's say we have an input RGB image with a resolution of $256 \times 256$ and $10$ classes:
(256 \times 256 \times 3) \rightarrow (256 \times 256 \times C_1) \rightarrow (256 \times 256 \times C_2) \rightarrow (256 \times 256 \times 100) \rightarrow (6553600) \rightarrow 10
Note how the image preserved its spatial structure throughout the convolutional part of the network and lost this only to be classified by the
fc layer. Now what does each convolution layer do? It pretty much extracts spatial features from the input image. The last layers of the network extract high-level features from the input image. The last convolutional layer produces $100$ feature maps for each input image.
The approach focuses replacing the flattening operation with a Global Average Pooling one, which averages each feature map to get a single number. The
fc layer then learns to classify an image just from those $100$ numbers!
(256 \times 256 \times 3) \rightarrow \; ... \rightarrow (256 \times 256 \times 100) \rightarrow (100) \rightarrow 10
The each number however represents a feature map. By taking a specific class (out the $10$) and seeing its weights from the
fc layer, we can tell which of the $100$ feature maps activates it the most. Furthermore, by applying the whole linear combination as designated by that class' weights we can produce what is referred to as the Class Activation Map (CAM) of that image. Through this we can tell what part of an image activates the CNN for a specific class. Furthermore, this is a very convenient way for a CNN to provide a reasoning for its decision, besides just the classification.