Multiple Non-linear Regression with Function-based regression and Machine Learning models

I'm working on an application of Multi-nonlinear regression. Initially, I tried this algorithmically by creating a polynomial of the form A(x^p * y^q * z^r). I saw poor results (highest r-squared value = 0.51), as shown below.

FORMAT: Output(x,y) = A*(1^2) + B*(1^1 * x^1) + C*(1^1 * y^1) + D*(x^2) + E*(x^1*y^1) + F*(y^2)

Notice: The powers of the variables always add up to 2.

Machine Learning

Seeing poor results here, I turned to machine learning applications. This produced only 70% accuracy (see below).

• Here's the code:
model = Sequential()
np.random.seed(3)
saveModel = ModelPerformancePlot("D:\\models\\model_{epoch:02d}.hdf5",
input_test, output_test, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1)
model.fit(input_train, output_train, validation_split = 0.50, epochs=256,
batch_size=64, verbose=2, callbacks = [saveModel])

• Here's the visual performance:

I understand accuracy isn't a good indicator of regression, I just added it to the graphic for visual effects.

My question:

Is there a way of improving the Machine Learning algorithm to fit better? Any suggestions on which layers are best for multi-nonlinear regression? What other options are there to Sequential.compile()'s metrics parameter? I do not want to use accuracy. The documentation is unclear here.

• Try adding a constant term "+ c" to the original regression, that should help. – James Phillips Jun 12 at 15:34
• @James Phillips Thanks for your comment. I actually already have a "+ C". I've updated the post to show the regression format better. I've been chewing on an idea: breaking up the daily equation into three parts. If you look at the top graph, there is an exponential rise at the start of the day, a polynomial fit in the middle, and an exponential decrease at the end of the day. Do you think it's possible to split regression like that? Interesting stuff... – BeardedDork Jun 12 at 16:03
• If you do this, i recommend some small data overlap between the differently modeled regions. This should help ensure smoother transitions between sections than if this were not done. – James Phillips Jun 12 at 16:27
• Thank you for zunzun.com, BTW. I used it a few weeks ago and it was really cool. Although, I didn’t get the same results when implementing the fit polynomials in Python. – BeardedDork Jun 12 at 16:32
• You are too kind, Oh Bearded One. – James Phillips Jun 12 at 18:09