1
$\begingroup$

I am fairly new to machine learning, so I apologize if this is a bad and/or repeat question. It does seem like questions of this nature have been asked at least for linear relationships. Let's say that I have a collection of noisy data that roughly resembles some function. For clarity's sake, take this function to be sine, so that we have data that goes like

$$f(x)=a\sin(bx)+\epsilon(x),$$

where $\epsilon(x)$ is some noise function that generates the noise in the data and $a,b$ are two parameters. Now suppose that we have no idea what a sine function is. That, or we know what it is, but we don't know the values of $a$ or $b$ that would allow us to train a neural network or BDT to fish out the sine function from all of the noise.

This is where my question comes in. Is there a way that I could use ML to still find a curve that models the data (and, if we are lucky, it happens to approximate sine from the noise correctly without knowing that it was a sine to start off with)?

Secondary question: If it happens to not be possible, is there a way to use ML to estimate $a$ and $b$ without resorting to conventional curve fitting techniques, such as least-squares? This is primarily a curiosity as to whether ML methods can be used to search the space of parameters for a given function (model) that best describes the data (perhaps more efficient, better at finding correlations in the data that would be typically accounted for via the use of a covariance matrix if one used something like least-squares, etc.).

$\endgroup$
2
  • $\begingroup$ So to clarify, you're asking how to use ML to estimate the parameters for this model? $\endgroup$ – Demetri Pananos Jun 12 '20 at 16:45
  • $\begingroup$ That was the secondary question that I added that the end. That would be extremely nice to know. It has been quite difficult sifting through the rich literature of ML to find an answer, especially since I am still learning the language that practitioners use. $\endgroup$ – user107053 Jun 12 '20 at 17:16
0
$\begingroup$
  1. I believe the response by @Sycorax-says-reinstate-monica offers the best advice. But, to answer your question directly, I believe it is possible to achieve what you describe with symbolic regression. Unlike standard linear regression, symbolic regression does not start with a mathematical model. It searches over the space of possible mathematical models to find the one which best explains the data. Therefore, it can determine that the sine function explains the data better than a polynomial, exponential, or anything else and therefore choose to model the relationship as such without you telling it.

  2. Curve fitting is all about estimating $a$ and $b$ so I'm not sure why you want to avoid it. Perhaps you can elaborate.

$\endgroup$
2
  • $\begingroup$ 1.) I've looked at it a little bit and I can honestly say that is so amazingly cool. Thank you! 2.) It is a bit more of a curiosity whether or not it can be tackled by ML methods. All of the ML methods that I have gotten my hands on are capable of learning a function from noisy data if you know the functional form without any free parameters. It would be interesting to know if there's some way I can use ML to perform a curve fit (i.e., find the parameters of a pre-specified model). $\endgroup$ – user107053 Jun 12 '20 at 18:32
  • $\begingroup$ I've edited to question to elaborate on it a bit more. $\endgroup$ – user107053 Jun 12 '20 at 18:50
1
$\begingroup$

Since this is a generic machine learning problem, there are any number of methods. A good place to start would be reading a high-quality textbook such as Elements of Statistical Learning.

If you know the form and want to estimate the parameters, you should look into nonlinear least squares. If you don't want to use least squares, you could use a different loss like $\log \cosh$.

$\endgroup$
1
  • $\begingroup$ Thank you for your suggestion! $\endgroup$ – user107053 Jun 12 '20 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.