# Adding Regressors to Convolutional Neural Network

I have an idea for working with geospatial rasters to predict housing prices. I have done this before using mulitivariate linear regression. I would like to try using a convolutional neural network. I have a series of rasters that represent a surface of interpolated housing prices over time. The convolutional neural network will learn from the price rasters, but I need to add in other explanatory variables. The most important is time. I might add additional scalar values such as interest rate or stock market. I might also want to add additional rasters as regressors, such as crime maps, or distance to transit. I know that in a typical neural network regression there is one neuron per explanatory variable, and in a typical CNN there is a neuron per pixel on the input layer. So I would like some regressors to be rasters, and some to be scalers.

My question is how do I add additional explanatory variables, both rasters and scalars, to a convolutional neural network? I am using Keras and TensorFlow.

## 1 Answer

It seems like you could do this relatively easily using a model that has MLP and CNN parts.

Suppose that your time-series data is $A$. You compute some CNN output $c(A)$ that uses $A$ as the input; this can return a vector since you have, say, $i$ units in the final layer of this portion.

Suppose that your "tabular" data is $B$. You compute some MLP that has output $m(B)$; this can return a vector since you have, say, $j$ nodes in the final layer of this MLP.

Now you have two vectors which, in some sense, encode the data contained in the tabular and CNN components of your data. You can concatenate these vectors to make a new vector of length $i+j=k$. This is the input to another fully-connected layer in your network, or possibly more than one. Then the output is just whatever your usual output is.

The reason that I think this could work is that you process the CNN and tabular pieces with models which are appropriate for their respective types, and then combine the results in a way which permits both formats to be used together.

But this could be hard to train. This would not be my first choice of a model. Instead, I would prefer to try using either a tabular or a CNN model by itself, and making a determination of whether or not either simpler model is suitable.

Note that this structure trains all parts of the network, the MLP, the CNN and the "combiner" part, all at once. This does not require you to train three separate networks.

This is easy enough to do in most modern neural network software, such as Keras.