I am looking for something like a linear regression but instead of the equation being linear it can be completely arbitrary.


For the equation of gravity, we already know $F = \frac{G m_1 m_2 }{ r^2}$

What I want is to feed an algorithm with a tabular set of values like this

$ \begin{array} {|r|r|} \hline F & m_1 & m_2 & r \\ \hline 1 & 2 & 3 & 4 \\ 5 & 2 & 5 & 4 \\ 1 & 3 & 2 & 4 \\ 1 & 2 & 5 & 4 \\ \hline \end{array} $

and it should try to predict the equation to the best of its abilities.

What is this form of regression called? Is there any algorithm that exists, which I can use?

  • 2
    $\begingroup$ The situation is not sufficiently specified. There are different possible "forms of regression" that might apply, depending on your model for the variation about the model; which variables have 'error', whether the variation is constant or related to the mean, and even on the form of the conditional distribution of the response. $\endgroup$ – Glen_b Apr 7 at 13:13
  • 1
    $\begingroup$ We are trying to predict the model from a set of data. No assumptions are made. For starters lets assume there are no errors. All variables in RHS are independent. $\endgroup$ – Souradeep Nanda Apr 7 at 13:20
  • 1
    $\begingroup$ If I understand you correctly, you want an algorithm that takes as input features m_1, m_2, r and the prediction target is F. In this case you can train a decision tree regressor, which can perfectly predict your target in the training sample. Note though that you usually do not want to have an estimator with a perfect training fit, because this estimator is likely to predict poorly on unseen data due to overfitting. $\endgroup$ – kanimbla Apr 7 at 14:14
  • 2
    $\begingroup$ You said the function can be completely arbitrary, and you don't want to make any assumptions. Now you're saying the function has to be "interpretable" and "nice and clean". These requirements are conflicting. $\endgroup$ – The Laconic Apr 7 at 15:09
  • 1
    $\begingroup$ $F = \frac{G m_1 m_2 }{ r^2}$ is not a polynomial $\endgroup$ – Souradeep Nanda Apr 7 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.