# Finding clusters to fit least squares and produce a piecewise equation

In the figure below, I've manually drawn an approximate solution to a least squares fit of the associated regions separated by black lines. The data appears to be bounded by two asymptotes (y=-18 and y = -75), which I then use for locating the regions.

1) What is a method for find these asymptotes and the associated three regions?

2) Is there a special name for this type of piecewise equation, which will be the result of LS fit on the data in each region?

• stat.ethz.ch/R-manual/R-patched/library/stats/html/loess.html Yes, it's called LOESS regression. Mar 3, 2014 at 1:07
• @EricPeterson Do you mean, use LOESS instead of piecewise regression? Or is there a option of R LOESS that does piecewise regression?
– xan
Mar 4, 2014 at 0:32

There is something called segmented regression, but that expects you to determine the breaks up front. To get the breakpoints determined for you, I think you need some kind of formula that incorporates them. Then you can use that in a nonlinear regression tool.

There is a package in R for that called segmented -- tutorial at R-bloggers. In SAS, you can use proc nlin: tutorial at UCLA site. I do it the hard way below, which is hopefully informative.

Before that, I should say I don't have an answer to your first question, except to note that the presence of Y asymptotes suggests something is going on that a simple regression which assumes random error won't take into account. Possibly the data is clipped or behaves with a logistic nature (S curve):

The asymptotes of this logistic fit gives values very close to your estimates: -18.4 and -75.3. I am using data scraped from your image, which will of course introduce errors in precision and possibly in omission due to overstriking.

Back to the second question, to make an equation that encapsulates the break points, you can either use an if-else form (as in the SAS example) or sum three line equations with weighting filters. The latter is fuzzier and more complex but is nicer to some nonlinear optimizers because the functions have smoother derivatives. (Because the solution is not usually findable analytically, nonlinear solvers use iterative optimization techniques to search the problem space.)

One way to write a smooth square function between two values a and b is as the difference of two logistic step functions:

1 / (1 + e^( -10 * (x - a) )) - 1 / (1 + e^( -10 * (x - b) ))


where 10 is a semi-arbitrary constant controlling how steep the step is. A graph:

For your problem, we can use three line equations, each multiplied by such as step function so the line is multiplied by 1 in the interval and 0 elsewhere. I tried this form:

Using m1, m2, m3 and the slopes, b1 as the initial y intercept and x1 and x2 as the two break points. For the three line parts, the lefthand side is the filter function between break points and the righthand side represents the line equations. The 2nd and 3rd line equations are more complex because they are constrained to connect to the previous line at the break points. The last line is to force x1 < x2.

Text form:

(1/(1+e^(-10 * (:x - 0))) - 1/(1+e^(-10 * (x - x1)))) * (b1 + m1 * x)
+ (1/(1+e^(-10 * (x - x1))) - 1/(1+e^(-10 * (x - x2)))) * (b1 + m1 * x + (m2-m1) * (x - x1))
+ (1/(1+e^(-10 * (x - x2))) - 1/(1+e^(-10 * (x - 150)))) * (b1 + m1 * x + (m2-m1) * (x - x1) + (m3-m2) * (x - x2))
+ e^(x1-x2)


Running this through a nonlinear solver with initial break values at 50 and 100 gives:

b1  -88.42
m1    0.41
m2    0.75
m3    0.18
x1   62.77
x2  110.44

• Extremely informative perspective. I ended up using this Matlab logistic regression function for prediction and finding the asymptotes. Should anyone try using this function, one should note the x inputs are required to be sorted and the y inputs must be positive.
– Rich
Mar 4, 2014 at 6:37