# How to estimate an upperbound for logistic regression by only 5 to 7 data points?

I have data that is of the form $y = \frac{\beta_1}{1 + \exp(\beta_2 + \beta_3 * x)}$. For the estimation of $\beta_1$ to $\beta_3$ I use the formulas of this paper: John Fox - Nonlinear Regression and Nonlinear Least Squares In this paper, $\beta_1$ is estimated by looking at the data. If I do that it works fine, even if I have only three points. From that I can calculate the two others. I tested my parameters with nls() in R and LevenbergMarquardt in C#. The models returned by them are satisfying.

The problem is I don't want to look at the data to get a good estimator for $\beta_1$, I want my program to calculate it. For some time I used values that were a bit higher than the maximum of my values (something between $\max * 1.1$ and $\max * 1.5$. This worked fine as long as the points covered most of the function. It also worked fine if the data points were somewhere from the "top" of the curve, but when they all came from the area "below" the inflection point, this estimator was definitely lower than it should be and I couldn't fit the model. If I use something that is definitely higher than the maximum point (by multiplying it with ridiculously high values) the model doesn't fit in any useful way.

Measurments could look like this:

x = (40, 50, 60, 70), y = (1000, 950, 400, 200) -> easy to estimate

x = (40, 50, 60, 70), y = (1000, 950, 800, 100) -> easy to estimate

x = (40, 50, 60, 70), y = (500, 200, 100, 50) -> not so easy to estimate

I guess I could find out where I am in the function (at the "bottom", at the "top", in the slope) by calculating the deltas in the given points and calculate an upper bound depending on that. Does anyone have a hint for a better solution? Additional info: If it can't be done, it's more important for me that measurements that can be fit are as good as possible and I accept that some measurements can't be fit at all.

(although I want an implementation in C# I posted it here, I don't think the problem is language dependent)

update (Application of this):

x are temperature values und y the corresponding measurements. By nature it is given that it should look like a logistic curve with higher y values at lower temperatures and vice versa. The melting point is equal to the inflection point of the curve, which changes a lot with small changes of the model parameters.

update (some made up data with 7 data points and known inflection point at 60):

//first I made up some data without any noise and enough (13) points
double[] x17 = { 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90 };
double[] y17 = { 2000, 1920, 1820, 1700, 1500, 1280, 1000, 720, 500, 300, 180, 80, 0 };
//calculated inflection point: 59.642....

//Then I took three different parts of that data
(to simulate how much data I will have in the real world)
double[] x18 = { 30, 35, 40, 45, 50, 55, 60 };
double[] y18 = { 2000, 1920, 1820, 1700, 1500, 1280, 1000 };
//calculated inflection point: 59.263.... is ok!

double[] x19 = { 60, 65, 70, 75, 80, 85, 90 };
double[] y19 = { 1000, 720, 500, 300, 180, 80, 0 };
//calculated inflection point: 53.447.... to small!

double[] x20 = { 45, 50, 55, 60, 65, 70, 75 };
double[] y20 = { 1700, 1500, 1280, 1000, 720, 500, 300 };
//calculated inflection point: 59.868... almost perfect!


My current approach for an upper bound estimation is y0 * y0 / y1. This way I hope to take into account that I might not always have a value close to the maximum.

update: The x and y values will never be negative. x will always be { 40, 45, 50, 55, 60, 65, 70 } unless a data point is lost.

update: I did many, many tests with simulated data that should be easy to fit (I choose data points that were exactly on the curve = without any noise) and I see that it works fine, unless the first or second data point (where x=40 or x=45) is missing. I guess I will have to discard such measurements and the user will have to live with that.

• With such a small number of points I think your real problem is the inaccuracy of the estimates and not how you calculate them. Commented Sep 11, 2012 at 15:49
• The problem with the last example is that a much better fit is obtained by a function of the form $y = \frac{\beta_1}{1 - exp(\beta_2 + \beta_3 * x))}$ (notice the negative sign). Is that an acceptable solution? If not, you will continue to have problems, because you will have to accept solutions on the boundary between the two models (and those are awful).
– whuber
Commented Sep 11, 2012 at 19:29
• @MichaelChernick I guess you are right, the problem is I will never get more than 7 data points. I guess I have to kick out results with less than that. I added test cases with results that should be the same. Is there room for improvement or is this the closest I can get? Commented Sep 12, 2012 at 6:51
• @whuber I tried to fit the data with the negative sign but for each parameter set I got an singular gradient matrix error (in R). Which estimators did you take? I added information about the area I use it in, to underline why the curve has to look that way. (I'm not sure yet if your formula satisfies these requirements) Commented Sep 12, 2012 at 6:54
• (1) What units are the $y$ values in? If these could be negative, then your model probably won't work. (2) What can you tell us about the nature of the "noise" or errors in the $y$ values? This can be crucial for fitting.
– whuber
Commented Sep 12, 2012 at 15:46

I worked on a similar problem earlier this year. The solution that I used was based on "Fitting a Logistic Curve to Data" by David Arnold (February 24, 2002), which uses linear algebra to eliminate $$\beta_1$$, the maximum. In this way, the results for the remaining parameters, once determined via some procedure (I simply minimized the error surface, same as in the paper), can be used to infer $$\beta_1$$ at the end.