Neural networks to predict a nonlinear curve [closed]

Question

I want to model a complex nonlinear function using neural networks (keras).

Training data: input - 8500 x 176 matrix of features, output - 8500 x 8 matrix, each row corresponds to 8 points which comprise a curve (nonlinear function of time, x-axis values are fixed at every 15 minutes between 0 and 2 hours). I need: to predict a curve given a single set of 176 features.

To my surprise, I failed to find in the web how similar problems should be addressed. I see three ways:

1) Make a network with 8 neurons in the last layer which represents the 8-point curve output. The curve points are correlated with each other -- will it not be a problem for ANN?

2) Add time as a separate feature and make (8500*8)x(176+1) feature matrix with a single value as an output. I suppose, this is essentially a multivariate curve fitting. Should I train the network so that all rows corresponding to a single curve end up in the same training batch?

3) Parametrize the curve, e.g. with cubic splines (the typical form of the curve is such that cubic splines should do reasonably well). What is the ANN topology? The second to last layer consists of neurons that correspond to spline parameters and the last layer is 8 neurons connected to the spline layer via a fixed formula? How should I implement that in keras?

Any general advises on ANN topology etc. are also very welcome. I suppose, with this kind of data it's better to use a multilayer perceptron. Am I wrong?

Answer 1

Essentially what you want is to change your input with 176 features into 8 values. For this you seem to have 8500 examples. If this understanding is incorrect, it means that the order of the data matters, which would mean you have to introduce a time component. If that is the case the problem becomes more involved.

For now I will assume your data does not have a time component that matters. The input should consists of your 176 characteristics and the output of 8 neurons. What you do in between is the way your architecture is buildup. There can be several layers with different activations. What works best for your problem is a hard task in general and practically impossible for us to answer since we don't know more about your data. A multilayer perceptron would be a good first approach though since it is simple to build and interpret the results.

The main point I want to give you is that much is dependent on your problem: what do you want to minimize/maximize, what is the corresponding loss function, what impact do you expect for certain variables? These kind of questions allow you to choose a logical set-up for your neural network.