I want to fit a curve $f(x) = mx+b$ on my data points $x_1, \ldots, x_N$ using linear regression with a single predictor.

However, the cost function is not even, rather, it has different weights on each side, i.e.:

$$ E = \frac{1}{N}\sum_{i=1}^N{\text{Cost}(f(x_i) - y_i)} . $$

where: $$\text{Cost}(v) = \begin{cases}v^2 &v\leq 0 \\\alpha \cdot v^2 & v> 0\end{cases} $$$$ \alpha > 0 $$

Are there any well-known methods for finding such a line for an arbitrary $\alpha$ value

I specifically wonder how should I find the line $mx+b$ that is totally under the points? (i.e. $\alpha=\infty$)

  • 4
    $\begingroup$ I believe this is what is called asymmetric least squares or expectiile regression. See, for instance, freakonometrics.hypotheses.org/files/2017/05/erasmus-1.pdf . Perhaps the R package expectreg an handle this, $\endgroup$ – Mark L. Stone Jul 9 '18 at 0:37
  • $\begingroup$ I am also interested. What packages are there for doing local (polynomial) quantile and expectile regressions in R? $\endgroup$ – Viktor Jul 9 '18 at 2:02
  • $\begingroup$ Re your last question: simply choose any $(m,b)$ that works! For instance, $m=0$ and $b=\min(y_i)$ obviously qualifies. Concerning the general question: first apply Calculus to solve the problem for the model $f(x)=b$ so you can understand the nuances. $\endgroup$ – whuber Jul 9 '18 at 12:34
  • 1
    $\begingroup$ @whuber That is indeed not correct, because we still need to minimize Cost(v) on for positive error values. $\endgroup$ – Ali Jul 9 '18 at 14:00
  • 1
    $\begingroup$ @whuber Thanks. Instead of setting $\alpha=\infty$, can we convert it into a constraint on the function? e.g.: Minimize $\sum{Cost((f(x_i)-y_i)^2)}$ s.t. $f(x_i) \leq y_i$ $\endgroup$ – Ali Jul 10 '18 at 10:25

Here is example code using the Python scipy.optimize.differential_evolution genetic algorithm module implementing a "brick wall" that gives a very large error if the genetic algorithm finds parameters that yield any predicted value above or below that of any data point per a code switch. It works in my tests when I flip the "upper/lower" switch in the code.

import numpy as np
import matplotlib
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
import warnings

from scipy.optimize import differential_evolution

xData = np.array([5.0, 6.1, 7.2, 8.3, 9.4])
yData = np.array([ 10.0,  18.4,  20.8,  23.2,  35.0])

def func(data, a, b):
    return a * data + b

# function for genetic algorithm to minimize (sum of squared error)
# this contains the "brick wall" switch for upper/lower
def sumOfSquaredError(parameterTuple):
    warnings.filterwarnings("ignore") # do not print warnings by genetic algorithm
    val = func(xData, *parameterTuple)
    for i in range(len(val)):
        if val[i] < yData[i]: # ****** upper/lower switch ******
            val[i] = 1.0E10
    return np.sum((yData - val) ** 2.0)

def generate_Initial_Parameters():
    # min and max used for bounds
    maxX = max(xData)
    minX = min(xData)
    maxY = max(yData)
    minY = min(yData)

    parameterBounds = []
    maxSlope = (maxY - minY) / (maxX / minX)
    parameterBounds.append([-maxSlope, maxSlope]) # parameter bounds for a
    parameterBounds.append([-maxY, maxY]) # parameter bounds for b

    # "seed" the numpy random number generator for repeatable results
    result = differential_evolution(sumOfSquaredError, parameterBounds, seed=3)
    return result.x

# generate initial parameter values
geneticParameters = generate_Initial_Parameters()

# create values for display of fitted peak function
a, b = geneticParameters
y_fit = func(xData, a, b)

plt.plot(xData, yData, 'D') # plot the raw data
plt.plot(xData, y_fit) # plot the equation using the fitted parameters

print('parameters:', geneticParameters)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.