I'm working on an application of Multi-nonlinear regression. Initially, I tried this algorithmically by creating a polynomial of the form A(x^p * y^q * z^r). I saw poor results (highest r-squared value = 0.51), as shown below.

FORMAT: Output(x,y) = A*(1^2) + B*(1^1 * x^1) + C*(1^1 * y^1) + D*(x^2) + E*(x^1*y^1) + F*(y^2)

Notice: The powers of the variables always add up to 2.

Algorithmic regression

Machine Learning

Seeing poor results here, I turned to machine learning applications. This produced only 70% accuracy (see below).

  • Here's the code:
model = Sequential()
model.add(Dense(128, input_dim=2, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(1, activation='linear'))
model.compile(loss='mse', optimizer='adam', metrics=['accuracy'])
saveModel = ModelPerformancePlot("D:\\models\\model_{epoch:02d}.hdf5", 
            input_test, output_test, monitor='val_loss', verbose=0, save_best_only=False, save_weights_only=False, mode='auto', period=1)
model.fit(input_train, output_train, validation_split = 0.50, epochs=256, 
          batch_size=64, verbose=2, callbacks = [saveModel])
  • Here's the visual performance:

I understand accuracy isn't a good indicator of regression, I just added it to the graphic for visual effects.

Machine Learning regression

My question:

Is there a way of improving the Machine Learning algorithm to fit better? Any suggestions on which layers are best for multi-nonlinear regression? What other options are there to Sequential.compile()'s metrics parameter? I do not want to use accuracy. The documentation is unclear here.

  • $\begingroup$ Try adding a constant term "+ c" to the original regression, that should help. $\endgroup$ – James Phillips Jun 12 at 15:34
  • $\begingroup$ @James Phillips Thanks for your comment. I actually already have a "+ C". I've updated the post to show the regression format better. I've been chewing on an idea: breaking up the daily equation into three parts. If you look at the top graph, there is an exponential rise at the start of the day, a polynomial fit in the middle, and an exponential decrease at the end of the day. Do you think it's possible to split regression like that? Interesting stuff... $\endgroup$ – BeardedDork Jun 12 at 16:03
  • $\begingroup$ If you do this, i recommend some small data overlap between the differently modeled regions. This should help ensure smoother transitions between sections than if this were not done. $\endgroup$ – James Phillips Jun 12 at 16:27
  • 1
    $\begingroup$ Thank you for zunzun.com, BTW. I used it a few weeks ago and it was really cool. Although, I didn’t get the same results when implementing the fit polynomials in Python. $\endgroup$ – BeardedDork Jun 12 at 16:32
  • $\begingroup$ You are too kind, Oh Bearded One. $\endgroup$ – James Phillips Jun 12 at 18:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.