How to evaluate fit of a logistic regression

I have a set of data points, which exhibit a solid linear correlation $r\approx 0.9$. I am basically plotting population in certain areas against the number of occurrences of a certain phenomenon (so in other words, I think the number of people should predict the number of occurrences of this phenomenon).

I don't like the linear regression though, because a linear model can take on negative or huge values. In my case the values need to be positive, and there is an upper bound (which I don't know; but logically, the values can only get so large). Unfortunately one of the inputs for which I need a predicted output is far larger than the inputs which I used to make a regression.

This made me choose a logistic model instead, $$y=\frac{A}{1+Be^{-Cx}}$$ because I am familiar with its shape (always positive, and approaching a limiting value) from basic differential equations, and it seems to be what I want for this situation. Also the data has a slight "point of inflection", so the curvature of the logistic regression (visually to me) is even a little better fit for the data points than the line was.

So finally my question(s):

1. I keep reading on the internet that logistic models are meant for probabilities, which take values between $0$ and $1$. I would like, in addition to having a high $r$, to try and reason why this type of model "makes sense" for my situation: am I going the completely wrong direction here?

2. How can I measure the "goodness of fit" of my curve to the data points? I have read Wikipedia but it is too vague for me to understand. An explicit formula - must be computable by hand calculator - would be helpful.

I have done my research but absolutely nothing I have found on the internet has been accessible to me, since I know virtually nothing about stats. An introductory/easy explanation would be so nice... and my math background is far stronger than my stats background, so use all the math you want but assume I'm a beginner at stats. I want it to be somewhat rigorous too - like if I could plug a number into a t-Test for example, that would be good to get an "objective" value.

• There are a lot of other generalized linear models and some transformations you could consider to model a lower-bounded distribution, and some are upper-bounded too if that's important. Check out beta and gamma regression and the logarithm transformation. Goodness-of-fit-wise, there are pseudo-$R^2$, the Akaike Information Criterion, and the Bayesian Information Criterion, among others...make it a structural equation model and you'll have plenty of other fit indices to choose from too (CFI, RMSEA, SRMR, etc.), but I don't know about calculating any of these by hand...Could be a headache. – Nick Stauner Jul 24 '14 at 4:21
• Before I look into other models, I would like to have a more precise idea of how good a fit my logistic regression is. And I have a good graphing calculator, with which I can do some basic programming operations, so all I need is an explicit formula. – user45240 Jul 24 '14 at 12:17
• Consider Generalized R squared, possibly with the Nagelkerke correction; or AUROC for standalone measures of model performance. You might also look through Frank Harrell's (he's a user here) posts on logistic regression validation, though he usually discusses model validation in terms of model selection. – user44764 Jul 25 '14 at 3:20

Standard univariate logistic regression of $y$ on $x$ finds the coefficients $\alpha$, $\beta$ that best fit your training data $\{(x_i, y_i), i \in [1, N]\}$ in the following equation:

(model 1): $y_i = (1 + exp(-(\alpha + \beta x_i)))^{-1}$

Note that the fit will be bad if the $y$ in your data are not in $(0,1)$, so you'll have to transform your data if you want to use logistic regression. One option might be to transform $y$ into a proportion (number of occurences of the "phenomenon" divided by population of the corresponding area?).

Also, the fact that "one of the inputs for which I need a predicted output is far larger than the inputs which I used to make a regression" is a problem, because you will be extrapolating the results of the model to unknown regions of the data. A value of $x_i$ much higher than the ones in the training sample will probably give you a $\hat y_i$ very close to 1.

Directly assessing prediction error

Once the model is fitted and you have your estimated parameters $\hat \alpha$ and $\hat \beta$, you get a predicted output value $\hat y_i$ for each observation $x_i$: $\hat y_i = (1 + exp(-(\hat \alpha + \hat \beta x_i)))^{-1}$. You can easily assess goodness of fit on your graphic calculator using the observed $y_i$ and their corresponding predict values $\hat y_i$:

• either by plotting one against the other (if the fit was perfect this would give you a straight line, the identity line, because then $y=\hat y$)

• or by computing an error measure, for instance the root mean squared error: $rmse= \sqrt{\frac{1}{N} \sum_{i=0}^N (y_i - \hat y_i)^2}$. This will tell the average distance between observed outcomes $y_i$ and the model-predicted outcomes $\hat y_i$ (the lower $rmse$, the better the fit). It is not a standardized score like $R^2$, but is easy to compute and interpret.

Now to asses the prediction power of the model, it is best to compare $y_i$ and $\hat y_i$ on a validation dataset, ie. data that were not used in the fit (eg. by withholding a portion of data during training, see cross-validation for more info).

Pseudo-R²

The usual $R^2$ of linear regression does not apply to logistic regression, for which several alternative measures exist. In all variants, $R^2$ is a real value between 0 and 1, and the closer to 1 the better the model.

One of them uses the likelihood ratio, and is defined as follows:

$R^2_L = 1 - \frac{L_1}{L_0}$, where $L_1$ and $L_0$ are the log-likelihood of (respectively) model 1 (see above) and the following model 0, which is a logistic regression on just a constant (and does not depend on $x$):

(model 0): $y_i = (1 + exp(-\alpha))^{-1}$

For any logistic regression model with $y \in \{0,1\}$ the log-likelihood is computed from the observed $y$ and the predicted $\hat y$, using the following formula (but I'm not sure it applies for continuous $y \in [0,1]$):

$L=\sum_{i= 1}^N(y_i \ln(\hat y_i)+(1−y_i)\ln(1−\hat y_i))$

Another pseudo-R² is based on the linear correlation of $y$ and $\hat y$, which is easily computed on any graphic calculator with stat functions:

$R^2_{cor} = \left( \widehat {cor(y_i, \hat y_i)} \right) ^2$

• Thank you for the answer! Is there any way that the values you give can be made more rigorous? I understand things like "smaller is better", but I would rather have a quantified version that was objective. – user45240 Jul 27 '14 at 19:14
• If you want a statistical goodness-of-fit evaluation, the natural way to go is the one suggested by Nick Stauner above, like using a pseudo-R2 en.wikipedia.org/wiki/…. These measures usually involve two measures of log-likelihood, one for your model and one for a model with no explanatory variables (just a constant). For a binary variable the log-likelihood is given by the following formula (but I'm not sure it applies for continuous $y$ in $[0,1]$): $logLik= \sum_{i=1}^N (y_i \ln(\hat y_i) + (1-y_i) \ln(1 - \hat y_i))$ – jubo Jul 27 '14 at 21:17
• That's the part of the Wiki page I linked to in my post, and I didn't understand it. – user45240 Jul 27 '14 at 22:51
• Thanks again - I awarded the bounty, but I would still like to get answers to my questions. – user45240 Aug 2 '14 at 19:54
• I edited my answer, hope it answers your questions. – jubo Aug 3 '14 at 19:36