Question about "curve fitting" using ML without known functional form I am fairly new to machine learning, so I apologize if this is a bad and/or repeat question. It does seem like questions of this nature have been asked at least for linear relationships. Let's say that I have a collection of noisy data that roughly resembles some function. For clarity's sake, take this function to be sine, so that we have data that goes like
$$f(x)=a\sin(bx)+\epsilon(x),$$
where $\epsilon(x)$ is some noise function that generates the noise in the data and $a,b$ are two parameters. Now suppose that we have no idea what a sine function is. That, or we know what it is, but we don't know the values of $a$ or $b$ that would allow us to train a neural network or BDT to fish out the sine function from all of the noise.
This is where my question comes in. Is there a way that I could use ML to still find a curve that models the data (and, if we are lucky, it happens to approximate sine from the noise correctly without knowing that it was a sine to start off with)?
Secondary question: If it happens to not be possible, is there a way to use ML to estimate $a$ and $b$ without resorting to conventional curve fitting techniques, such as least-squares? This is primarily a curiosity as to whether ML methods can be used to search the space of parameters for a given function (model) that best describes the data (perhaps more efficient, better at finding correlations in the data that would be typically accounted for via the use of a covariance matrix if one used something like least-squares, etc.).
 A: Since this is a generic machine learning problem, there are any number of methods. A good place to start would be reading a high-quality textbook such as Elements of Statistical Learning.
If you know the form and want to estimate the parameters, you should look into nonlinear least squares. If you don't want to use least squares, you could use a different loss like $\log \cosh$.
A: *

*I believe the response by @Sycorax-says-reinstate-monica offers the best advice. But, to answer your question directly, I believe it is possible to achieve what you describe with symbolic regression. Unlike standard linear regression, symbolic regression does not start with a mathematical model. It searches over the space of possible mathematical models to find the one which best explains the data. Therefore, it can determine that the sine function explains the data better than a polynomial, exponential, or anything else and therefore choose to model the relationship as such without you telling it.


*Curve fitting is all about estimating $a$ and $b$ so I'm not sure why you want to avoid it. Perhaps you can elaborate.
