0
$\begingroup$

Back in High School I remember the TI-84 calculator allowing the user to enter a few data points and then select from a list of options to find an equation that best fit the data points. One of these options was called Logistic and it found the variables "a","b", and "c" for the equation of the form

$$ y = \frac{1}{c+ae^{-bx}} $$

I figured that I would easily find an online calculator that would do the same thing, but after going through various google results for "logistic regression" or "logistic best fit", I found something with which I was not familiar. The online calculators for logistic regression seem to be for classifying data rather than provide a best-fit equation. Perhaps there's a way to use the calculators, but they all require a column with 0 or 1, which doesn't make sense for my data.

After further investigation, I was unable to find the source code or method by which the TI-84 was calculating the best-fit logistic equation to fit a set of data points. Also, I was unable to find any other online calculator that would give me the functionality of the TI-84 calculator.

If you're interested, the data points I have are as follows:

Hours Remaining
0 38708
1 38466
3 37444
3.5 37126
4 36642
4.5 36001
5 35275
5.5 34460
6 33079
7 29936
8.5 19587
9 12136

Now, this data wouldn't fit the original form, but rather a form like $$ y = d - \frac{1}{c+ae^{f-bx}} $$

Since I couldn't find any easy way to fit the data, I decided to guess using desmos.com and 5 sliders for the a,b,c,d,and f variables. After some guessing I was able to come up with $$ y = 40500 - \frac{1}{-0.00002+0.0000794e^{2.3-0.29x}} $$ which seems to fit somewhat okay.

My background is more on the software side of things, so I'm interested in knowing how a computer program might be able to solve this type of problem or at least find a close approximation.

Improve this question
$\endgroup$
4
  • $\begingroup$ logistic regression has 'c ' set to 1. a simple methodology is gradient descent ( change a parameter, does error go down...). You could try a google sheets "solver" workspace.google.com/marketplace/app/solver/539454054595 ? $\endgroup$
    – seanv507
    Jun 3, 2022 at 12:19
  • $\begingroup$ Starting with $y = \frac{1}{c+ae^{-bx}}$ you can get to $\log_e\left(\frac{cy}{1-cy}\right) = \log_e\left(\frac{c}{a}\right)+bx$ where the left-hand side is a log-odds or logit function of $cx$ and the right-hand side (a constant plus a multiple of $x$) is suggestive of linear regression, so the whole thing is suggestive of logistic regression. The $c$ term complicates matters slightly, and note you need $0 < cy < 1$ for all the observations of $y$ $\endgroup$
    – Henry
    Jun 3, 2022 at 12:55
  • $\begingroup$ With your $y = d - \frac{1}{c+ae^{f-bx}}$ I think you get $\log_e\left(\frac{(d-y)c}{1-(d-y)c}\right) = \log_e\left(\frac{c}{a}\right)-f+bx$ needing $y < d < y+1/c$ for all the observations of $y$ $\endgroup$
    – Henry
    Jun 3, 2022 at 12:57
  • 1
    $\begingroup$ The standard algorithm for minimizing least squares problems is Levenberg-Marquardt. Understanding its description, e.g. on Wikipedia or elsewhere, requires some knowledge in vector analysis: deriving a scalar function $f$ by a vector $\vec{x}$ yields the "gradient", i.e. the vector of partial derivatives $\partial f/\partial x_i$, and deriving a vector $\vec{f}$ by a vector yields the Jacobian matrix of all aprtial derivation combinations $\partial f_i/\partial x_i$. $\endgroup$
    – cdalitz
    Jun 3, 2022 at 13:53

2 Answers 2

Reset to default
0
$\begingroup$

Logistic regression is for a response variable with discrete classes (TRUE/FALSE). It's fit by the maximum likelihood method, which uses an assumed probability distribution of model errors to calculate the joint probability that the data could have been produced by the model.

Because "logistic regression" was one option of many in the calculator menu and this example is for continuous numeric data, I would guess that the calculator is applying a general purpose nonlinear optimization method to find the model parameters that minimize the sum of squares error. Unlike maximum likelihood, minimizing sum of squares error can be applied to any curve fitting problem without making deliberate choices about probability distributions. If this is true, it's not a true logistic regression, but rather curve fitting with nonlinear optimization applied to a logistic function.

The most commonly-used algorithms for minimizing the sum of squares error of a nonlinear function are gradient-based. They're actually not limited to minimizing model error; they can minimize any smooth function you can define. They require a starting guess for the parameters, calculate a direction to change the parameters to reduce error, and take a step in that direction, and iterate until a local minimum is found.

The most intuitive algorithm to accomplish this is steepest descent, which simply takes steps in the direction that most strongly reduces the error. It turns out that this isn't the most robust way to do it, so more commonly used algorithms are Conjugate Gradient and Levenberg-Marquardt.

Improve this answer
$\endgroup$
0
$\begingroup$

The model equation that you propose : $$y = D - \frac{1}{C+Ae^{F-Bx}} \tag 1$$ has a overabundant number of parameters (five) because two of them ($A$ and $F$) are dependent. If one uses this function in a software for nonlinear regression and fitting this might causes convergence failure.

The correct writting would be with only four parameters : $$y = D - \frac{1}{C+Ge^{-Bx}} \tag 2$$ because $G=Ae^F$ .

A method of fitting which doesn't requires iterative calculus is presented below.

Take care about the notations which are different. $$y = a + \frac{b}{1+c\:e^{-p\:x}} \tag 3$$ $$a=D \quad ; \quad b=-\frac{1}{C} \quad ; \quad c=\frac{G}{C} \quad ; \quad p=B \tag 4$$

enter image description here

Don't confuse the matrix notations $A,B,C,D,E,F,G$ with the notations in the equations $(1)$ , $(2)$ , $(4)$ .

Numerical example :

enter image description here

If the accuracy of fit is not sufficient one have to use a software for non-linear regression. The advantage is that we have found very good approximate values of the parameters that can be used as initilial values to start the iterative process. This is much more reliable than starting from guessed initial values.

REFERENCE : https://fr.scribd.com/doc/14674814/Regressions-et-equations-integrales This paper explains the principle of the method with a lot of numerical examples.

NOTE : The data doesn't includes the points at large $x$ low $y$. So the logistic function is badly defined for $x>10$ . The values of the parameters (an even the signs) cannot be trusted for large $x$. Possibly a simpler function of the exponential kind would be sufficient for a good fiting as well and would be less likely to mislead.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.