Fitting decaying exponential to binary response

Question

I have this data that I want to fit with $y = e^{-bx}$, but the y:s represent probabilities and the outcomes are either 0 or 1, so I can't say $ln (y) = -bx$ since the values will just alternative between -inf and 0. Is there a way to do this?

Answer 1

You can fit this model directly using a generalized linear model.

Since your data are 0-1, you would want the binomial family, and a $\log$ link. This is available in many packages.

However, you must be very careful with the log-link in a binomial model, since it's not constrained to lie in (0,1), it's only constrained at the 0-end -- so if the values for $P(Y=1)$ can get up near 1, the algorithm may have difficulty.

Here I've done an example in R.

First some data:

This is data generated from an exponential decay model in $P(Y=1|X=x)$, and (in purple) a lowess smooth performed on transformed $y$ (then backtransformed), so that we can see that locally it looks like typically it does progress as desired.

I jittered (added a tiny bit of noise to) the y-values for the plot so you could more easily see the relative density.

Here's the command I used to glm fit and the resulting output (with a few unnecessary lines removed), performed in R. You'll note that I supplied starting values; usually this isn't necessary for GLMs, but may be the case for your model (it was for this data - it had trouble finding good start values on its own). My strategy for getting start values was to first fit a model without intercept and then supply start values as 0 for the constant-term and the coefficient from the no-intercept model for the x-term, which was sufficient in this case. A more sophisticated strategy might involve taking information from the lowess fit to guess good start values, but that wasn't needed here. A third possibility (if the typical proportion is small) would be to use parameters from a logit link as starting values, since if the proportions are small, log and logit are similar.

 logfit=glm(y~x,family=binomial(link=log),start=c(0,-.227))
 summary(logfit)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  0.85492    0.16230   5.268 1.38e-07 
x           -0.38053    0.04049  -9.399  < 2e-16 

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 364.22  on 319  degrees of freedom
Residual deviance: 264.65  on 318  degrees of freedom
AIC: 268.65

Number of Fisher Scoring iterations: 5

---

The true coefficients in this case were 0.6 and -0.32; both estimated coefficients where within about 1.5 standard errors of the population values.

Here's the fit, in red (and for comparison, the model I generated the data from, in dashed dark-blue):

It's no big surprise that the fit drops more quickly than the true curve, since the local smooth we did at the beginning dropped faster still (it looks like there was an excess of 0's in the middle of the left-half, say around x=4 to 5, compared to what you might expect, causing a more rapid drop in the fit).

---

Advice wise, if you don't have a particularly good reason outside the appearance of the data itself to use an exponential decay, you might consider a logit-link model instead. It's less likely to stray into difficulty.

Answer 2

Ben -

You can fit $ln(y) = \beta x$ with a generalized linear regression (GLM) model called a Poissson regression. Better yet, since the observed y is either 0 or 1, one uses a logistic regression to estimate probability of y, where the logit function replaces the log: $$ logit(y) = \beta x$$