# Combining image and scalar inputs into a neural network

I was reading a paper that some Google researchers wrote (find it here: https://arxiv.org/abs/1603.02199) and having a look at their neural network, they have two images stacked on top of each other as one input and in the middle, they have a motor command (a series of scalars basically) tiled into the output of a max pool layer.

In their words: "The motor command is processed by one fully connected layer, which is then pointwise added to each point in the response map of pool2 by tiling the output over the special dimensions".

Can anybody explain how exactly this works? Or in other words, how would one insert a scalar or a series of scalars as an input to a CNN?

There are many ways to combine scalar and image inputs. In this particular paper, a diagram on the top of page 5 should explain everything. At some point in the convolutional network there are 64 feature maps, which matches the 64 scalar values to be input. The 64 scalar values are essentially treated as bias terms so that the $i$th scalar value is added on to the $i$th feature map.
• $i$th scalar value is added to all values of $i$th feature map, isn't it? – Saravanabalagi Ramachandran May 14 '18 at 15:12