Regression with "unidirectional" noise I would like to learn a function $f: \mathbb{R}^n \rightarrow \mathbb{R}$ from data $(x_i, y_i)$ where $y_i = f(x_i) + \epsilon_i$. $\epsilon_i$ can be seen as observation noise; however, it is always smaller or equal to 0 ($\epsilon_i \leq 0$), i.e., the observations are always a lower bound on the true function value. Are there any regression approaches particularly suited for this setting? Is it required to make specific assumptions regarding the distribution of $\epsilon_i$?
 A: This set up is equivalent to the Deterministic (Efficiency/Productivity) Frontier Analysis in Econometrics, where the econometrician is trying to measure how far a firm/unit of production is from full-efficiency in the utilization of production factors. The $f(x)$ function is the full-efficiency production function (i.e. it gives maximum output given technology and inputs $x$, the "production frontier") and the error embodies a measurement of the distance of actual output from the theoretical maximum,
$$q_i = f(x_i) + \epsilon_i,\;\; \epsilon_i\le 0$$
This model has been largely abandoned, because in it, one of the regularity conditions for maximum likelihood estimation is violated: since $\epsilon_i\le 0$ we have always
$$q_i \le f(x_i)$$
which makes the range (i.e. the support) of the random variable $q_i$ (actual production) dependent on the parameters to be estimated (that are included in $f(x_i)$). Then the standard asymptotic properties of maximum likelihood estimators cannot be invoked, i.e. it is unknown whether they hold or not.  
So it has been replaced by the Stochastic Frontier framework, where alongside the one-sided error-term, a zero-mean symmetric disturbance (usually assumed normal) is added (that represents chance effects on the output of the firm that are not related to the "internal efficiency" of the firm):
$$q_i = f(x_i) + u_i+\epsilon_i,\;\; E(u_i\mid x_i) = 0,\;\;\epsilon_i\le 0$$
which deals with the issue mentioned above (and it is after all, more realistic also). Can you augment your model also, by adding a symmetric zero-mean error? Then the machinery of ML estimation is already in place in the Stochastic Frontier literature, with more than one stochastic specifications worked out.
