# Indexing the formula for the ith layer of a linear deep neural network

In a paper about deep neural networks they say:

In a general feedforward linear network described by an underlying directed acyclic graph, units can be organized into layers using the shortest path from the input units to the unit under consideration. The activity in unit $i$ of layer $h$ can be expressed as:

$$S^h_i(I)=\sum_{l<h}\sum_jw^{hl}_{ij}S^l_j \,\,\,\,\,\,\,\mathrm{with}\,S^0_j=I_j$$

I'm a little confused by the upper indexing on $w$. What would it mean if $w$ had an upper index $(h,h-2)$, for instance?

The weight $w^{hl}_{ij}$ is simply denoting the strength of the connection between the unit $i$ in layer $h$ and the unit $j$ in some previous layer $l$. In most feed-forward networks without any skip/residual/shortcut connections, $w^{h,h-2}$ will be zero.
• This is taken from page 5 of the linked article. In the previous sentence (the first sentence of section 3), he says that which layer a unit A is in can be defined by the shortest path from the input layer to A. So doesn't this definition ensure that layer h could never be connected to layer $h-2$? Feb 17 '18 at 15:14