It is common to give some multivariate time series to a Neural Network and get predictions for each individual time series. But my question is, does the NN take all series in consideration when creating the forecasts for one series?

Here is an example:

When trying to predict the number of Deceased, Infected, Recovered and Healthy people in a country it is possible to give the following data to a NN.

      Deceased | Infected | Recovered | Healthy
Day 1      10  |       20 |        10 |      60
Day 2      15  |       30 |        15 |      40
Day 3      20  |       40 |        20 |      20
Day 4      25  |       50 |        25 |       0

A naïve model would output some results that follow the trend present in the data:

      Deceased | Infected | Recovered | Healthy
Day 5      30  |       60 |        30 |     -20

This NN produced an output for each series without taking each other into consideration. A more robust model would 'realise' that there cannot be a negative number of healthy people (or any king of people) and create an output like this:

      Deceased | Infected | Recovered | Healthy
Day 5      30  |       40 |        30 |       0

I have read about making the model always output positive values here and here. But this is different in the sense that I want to know how to make a NN take into consideration the other time series then making predictions.

Is a multivariate model simply making predictions for each sequence in parallel or is it taking all variables in a time step into consideration?


But my question is, does the NN take all series in consideration when creating the forecasts for one series?

Well, that depends your exact model. A typical choice might be a neural network $f$, with the model defined as: $p(x^k_t| x_{<t}; \theta) = \mathcal{N}(\mu^k, \sigma^k = f(x_{<t}, \theta))$ where $t$ indexes time, $k$ indexes over your 4 sequences, and $\mu$ and $\sigma$ are vectors of size 4. In this case, $x^1_t$ is independent of $x_t^2$ (conditioned on previous elements $x_{<t}$) so the model does not "take into account" the prediction for sequence 1 when predicting sequence 2.

In fact, the well known pixelrnn paper had basically the same issue. They modeled images as 3 sequences of pixel values (one sequence for red, one for green, one for blue). And this had the same issue -- the blue value of a pixel may depend in some way on the red value. So, their model takes this into account by defining: $p(x^k_t| x_{<t}; \theta) = \mathcal{N}(\mu^k, \sigma^k = f(x_{<t}, x_t^{<k},\theta))$, allowing the neural network to learn a joint distribution over the three pixel values (or in your case, the 4 categories).

P.S. actually I believe they used a discrete distribution in pixelrnn, and not a gaussian, but the point is the same.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.