I have a function $f: \mathbb{R} \to \mathbb{R}_+$ and I would like to estimate it. The data pairs $\{(x_i, f(x_i))\}$ arrive at different times $t$. I have two questions:

  1. In this case, since the codomain of $f$ is $\mathbb{R}_+$ is density estimation the same as function approximation?
  2. What techniques are available to approximate $f$ sequentially? I know GPs are an option but they are super expensive and probably an overkill for a 1D problem. Maybe some Gaussian Mixture Regression? I tried looking at the literature but I can't find anything that has an implementation in say python or julia.

Importantly, is this non-linear regression? What is this task called? I am so confused.

  • 1
    $\begingroup$ Just to clarify data arrives like this: - At time $t=1$ we get $x_1$ and compute $f(x_1)$. - At time $t=\tau$ we get $x_\tau$ and would like to UPDATE my previous function approximation $\hat{f}_{\tau-1}$ to $\hat{f}_\tau$ by using $(x_\tau, f(x_\tau))$ $\endgroup$ Apr 16, 2021 at 9:48
  • 3
    $\begingroup$ To estimate a real number or a function, the minimal requirement is a probabilistic framework. (And a positive valued function is usually not a density function. It may even have an infinite mass.) $\endgroup$
    – Xi'an
    Apr 16, 2021 at 10:57