2
$\begingroup$

I have this data that I want to fit with $y = e^{-bx}$, but the y:s represent probabilities and the outcomes are either 0 or 1, so I can't say $ln (y) = -bx$ since the values will just alternative between -inf and 0. Is there a way to do this?

$\endgroup$

2 Answers 2

3
$\begingroup$

You can fit this model directly using a generalized linear model.

Since your data are 0-1, you would want the binomial family, and a $\log$ link. This is available in many packages.

However, you must be very careful with the log-link in a binomial model, since it's not constrained to lie in (0,1), it's only constrained at the 0-end -- so if the values for $P(Y=1)$ can get up near 1, the algorithm may have difficulty.

Here I've done an example in R.

First some data:

enter image description here

This is data generated from an exponential decay model in $P(Y=1|X=x)$, and (in purple) a lowess smooth performed on transformed $y$ (then backtransformed), so that we can see that locally it looks like typically it does progress as desired.

I jittered (added a tiny bit of noise to) the y-values for the plot so you could more easily see the relative density.

Here's the command I used to glm fit and the resulting output (with a few unnecessary lines removed), performed in R. You'll note that I supplied starting values; usually this isn't necessary for GLMs, but may be the case for your model (it was for this data - it had trouble finding good start values on its own). My strategy for getting start values was to first fit a model without intercept and then supply start values as 0 for the constant-term and the coefficient from the no-intercept model for the x-term, which was sufficient in this case. A more sophisticated strategy might involve taking information from the lowess fit to guess good start values, but that wasn't needed here. A third possibility (if the typical proportion is small) would be to use parameters from a logit link as starting values, since if the proportions are small, log and logit are similar.

 logfit=glm(y~x,family=binomial(link=log),start=c(0,-.227))
 summary(logfit)

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  0.85492    0.16230   5.268 1.38e-07 
x           -0.38053    0.04049  -9.399  < 2e-16 

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 364.22  on 319  degrees of freedom
Residual deviance: 264.65  on 318  degrees of freedom
AIC: 268.65

Number of Fisher Scoring iterations: 5

The true coefficients in this case were 0.6 and -0.32; both estimated coefficients where within about 1.5 standard errors of the population values.

Here's the fit, in red (and for comparison, the model I generated the data from, in dashed dark-blue):

enter image description here

It's no big surprise that the fit drops more quickly than the true curve, since the local smooth we did at the beginning dropped faster still (it looks like there was an excess of 0's in the middle of the left-half, say around x=4 to 5, compared to what you might expect, causing a more rapid drop in the fit).


Advice wise, if you don't have a particularly good reason outside the appearance of the data itself to use an exponential decay, you might consider a logit-link model instead. It's less likely to stray into difficulty.

$\endgroup$
2
  • $\begingroup$ Hi and many thanks for mentioning link functions. I tried a couple of them and actually the cauchit seemed to fit best. I'm not sure I'm interpreting it right however - it seemed to have the best Pr(>|z|)-values aswell as the smallest residuals. Intuitively, a distribution with allowance for extreme values makes more sense aswell. Am I interpreting this right? $\endgroup$ Oct 18, 2014 at 12:27
  • $\begingroup$ I'd advise against link-function shopping. In particular, by doing so, your p-values are no longer meaningful, and you bring in a host of other problems. Better, where possible, to choose on the basic of subject-area understanding. If you really have no good idea of what link-function to use, then you must incorporate or otherwise deal with the impact of searching for a model on your inferences. $\endgroup$
    – Glen_b
    Oct 18, 2014 at 23:32
2
$\begingroup$

Ben -

You can fit $ln(y) = \beta x$ with a generalized linear regression (GLM) model called a Poissson regression. Better yet, since the observed y is either 0 or 1, one uses a logistic regression to estimate probability of y, where the logit function replaces the log: $$ logit(y) = \beta x$$

$\endgroup$
4
  • $\begingroup$ I'd wrap a $E()$ around $y$. $\endgroup$
    – Momo
    Oct 17, 2014 at 16:00
  • $\begingroup$ First, since $y$ is an indicator variable $E[y] = P(y)$. So, as you note, for prediction, $P(y) = logistic(\beta x)$, where the logistic() is the inverse of the logit(). But the equation as written is the correct model. Note that unlike a conventional linear regression, it is written without an error term. $\endgroup$
    – John Mark
    Oct 17, 2014 at 17:38
  • $\begingroup$ The reference to an "observed y" creates a distinct possibility of mis-reading your answer, John, because one is left wondering what each instance of "$y$" or "y" refers to: an observation or a probability? This may be what @Momo is getting at. I take it the appearances of "$y$" in the two equations refer to probabilities and otherwise the two other mentions of "y" refer to observations (because that would make the answer correct), but it would be nice to adopt a notation that makes that distinction--which is at the very heart of the original question--much clearer. $\endgroup$
    – whuber
    Oct 17, 2014 at 18:19
  • 1
    $\begingroup$ Given that $y \in \{0, 1\}$, it follows that $logit(y) = \log[y / (1-y)]$ is undefined when $y=1$. When $y=0$, $logit(y) = -\infty$. Bottom line: best wrap $y$ with $E()$ as pointed out by @Momo. $\endgroup$
    – ramhiser
    Oct 18, 2014 at 5:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.