# Fitting a curve OVER OR UNDER a set of points

I want to fit a curve $f(x) = mx+b$ on my data points $x_1, \ldots, x_N$ using linear regression with a single predictor.

However, the cost function is not even, rather, it has different weights on each side, i.e.:

$$E = \frac{1}{N}\sum_{i=1}^N{\text{Cost}(f(x_i) - y_i)} .$$

where: $$\text{Cost}(v) = \begin{cases}v^2 &v\leq 0 \\\alpha \cdot v^2 & v> 0\end{cases}$$$$\alpha > 0$$

Are there any well-known methods for finding such a line for an arbitrary $\alpha$ value

I specifically wonder how should I find the line $mx+b$ that is totally under the points? (i.e. $\alpha=\infty$)

• I believe this is what is called asymmetric least squares or expectiile regression. See, for instance, freakonometrics.hypotheses.org/files/2017/05/erasmus-1.pdf . Perhaps the R package expectreg an handle this, – Mark L. Stone Jul 9 '18 at 0:37
• I am also interested. What packages are there for doing local (polynomial) quantile and expectile regressions in R? – Viktor Jul 9 '18 at 2:02
• Re your last question: simply choose any $(m,b)$ that works! For instance, $m=0$ and $b=\min(y_i)$ obviously qualifies. Concerning the general question: first apply Calculus to solve the problem for the model $f(x)=b$ so you can understand the nuances. – whuber Jul 9 '18 at 12:34
• @whuber That is indeed not correct, because we still need to minimize Cost(v) on for positive error values. – Ali Jul 9 '18 at 14:00
• @whuber Thanks. Instead of setting $\alpha=\infty$, can we convert it into a constraint on the function? e.g.: Minimize $\sum{Cost((f(x_i)-y_i)^2)}$ s.t. $f(x_i) \leq y_i$ – Ali Jul 10 '18 at 10:25

Here is example code using the Python scipy.optimize.differential_evolution genetic algorithm module implementing a "brick wall" that gives a very large error if the genetic algorithm finds parameters that yield any predicted value above or below that of any data point per a code switch. It works in my tests when I flip the "upper/lower" switch in the code.

import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
import warnings

from scipy.optimize import differential_evolution

xData = np.array([5.0, 6.1, 7.2, 8.3, 9.4])
yData = np.array([ 10.0,  18.4,  20.8,  23.2,  35.0])

def func(data, a, b):
return a * data + b

# function for genetic algorithm to minimize (sum of squared error)
# this contains the "brick wall" switch for upper/lower
def sumOfSquaredError(parameterTuple):
warnings.filterwarnings("ignore") # do not print warnings by genetic algorithm
val = func(xData, *parameterTuple)
for i in range(len(val)):
if val[i] < yData[i]: # ****** upper/lower switch ******
val[i] = 1.0E10
return np.sum((yData - val) ** 2.0)

def generate_Initial_Parameters():
# min and max used for bounds
maxX = max(xData)
minX = min(xData)
maxY = max(yData)
minY = min(yData)

parameterBounds = []
maxSlope = (maxY - minY) / (maxX / minX)
parameterBounds.append([-maxSlope, maxSlope]) # parameter bounds for a
parameterBounds.append([-maxY, maxY]) # parameter bounds for b

# "seed" the numpy random number generator for repeatable results
result = differential_evolution(sumOfSquaredError, parameterBounds, seed=3)
return result.x

# generate initial parameter values
geneticParameters = generate_Initial_Parameters()

# create values for display of fitted peak function
a, b = geneticParameters
y_fit = func(xData, a, b)

plt.plot(xData, yData, 'D') # plot the raw data
plt.plot(xData, y_fit) # plot the equation using the fitted parameters
plt.show()

print('parameters:', geneticParameters)