# How to find the input layer and the architecture for a Neural Network

I'm a software developer and I'll like to learn about neural networks. At this point I've find a problem which I'll like to solve at some point. It is about electrical load forecasting. I'm looking for similar problems and it will be great if I can find some similar examples with solutions. At this point I'm having troubles in finding the right model for the RNN, and more exactly I'm struggling with the input layer. As the output I need the forecast values for each hour.

Any reference to books, links resources or advices are welcome and very appreciated.

This is the problem that I'll like to solve:

A very small factory, use a number of equipments to produce bread. Some of them are electrical equipments which means that they consume electrical power. Knowing which equipments will run on the next day, an electricity consumptions forecast can be computed.

The equipment named E.V. is a special case of equipment. The human operator completes it's values in an empirically manner in order to have a good forecast for the next day. Those values can be positive or negative. The 'on' thing from the bellow table represents the status of the machines.

+----------------------------------------------------+
|equipment name|power| 1h| 2h| 3h| 4h| 5h| 6h| 7h| 8h|
+----------------------------------------------------+
|Equipment 1   |  2MW| - | - | on| on| on| - | - | - |
|Equipment 2   |  5MW| - | - | - | on| on| on| - | - |
|Equipment 3   |  1MW| on| on| on| on| on| on| on| on|
|E.V.          |     | .1|-.1|-.1| .1|-.2| .1| .1|-.1|
+--------------+-------------------------------------+
|total/(forecast)    |1.1|0.9|2.9|8.1|7.8|6.1|1.1|0.9|
+--------------+-------------------------------------+
|real consumption    |0.9|0.9|2.7|8.2|7.9|3.1|0.8|0.7|
+--------------+-------------------------------------+


The problem is that the machines are not running at their maximal power, so it will be great if a more exactly forecast can be build.

I have data from 2 years back for every day. Also, do you think that date or week day is a good candidate for the input layer?

I'm not very efficient in understanding an answer with a math only approach. Any example with something more appropiated with my problem is verry appreciated

Define neural network to be $f$, time-series to be $x$, lag order to be $n$ and forecast horizon to be $h$.

$f(x_{t-1}, x_{t-2},..,x_{t-n}) = [x_t, x_{t+1},..,x_{t+h}]$

Assume you have the following time series,

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]


You define $n=2$, $h=1$.

Your inputs for that time-series are circulant matrix like.

x =

[[ 1, 0],
[ 2, 1],
[ 3, 2],
[ 4, 3],
[ 5, 4],
[ 6, 5],
[ 7, 6],
[ 8, 7]]


y =

[2, 3, 4, 5, 6, 7, 8, 9]


So the length of your input layer is given by $n$, the length of your output layer is given by $h$, where your first input neuron is $x_{t-1}$ and your last input in $x_{t-n}$. Same goes for the forecast horizon.

Instead of having multiple outputs for the forecast horizon, you can use a forecast horizon of 1 then recurse on the predictions to obtain any forecast horizon you want.

For classic parametric stationary time series models the limit of the recursive behaviour of the system is well-studied.

Your problem is a little more involved though. You have inputs and outputs of the system and you want the predict outputs to follow some reference trajectory.

One solution is to use Narma-L2, which approximates the system by linear feedback using two neural networks. Define control inputs to be $c$ and production outputs to be $p$. Define reference production outputs to be $r$

You train two neural networks of the forms $g(c_{t-1}, .., c_{t-n}, p_{t-1},..,p_{t-n}) = c_{t}$ and $k(c_{t-1}, .., c_{t-n}, p_{t-1},..,p_{t-n}) = p_{t}$.

The prediction for control inputs is then $c_t = \frac{r - k(c_{t-1}, .., c_{t-n}, p_{t-1},..,p_{t-n})}{g(c_{t-1}, .., c_{t-n}, p_{t-1},..,p_{t-n})}$

Also, neural networks are a PITA. There's plenty of good nonparametric regression models that are easier to train, like Gaussian Process Regression for instance.