0
$\begingroup$

We're using logistic regression to predict events probability. Logistic regression tries to minimize the residual variance (sum of squared residuals). However, in our specific problem we would like to use a different loss function (for the errors in the logistic regression). Does anyone knows any implementation (or how to implement) such change? (preferably in R or Python) Thanks!

$\endgroup$
10
  • 6
    $\begingroup$ The typical approach to fitting logistic regression models is via maximum likelihood. This is not equivalent to minimizing the variance of the residuals. $\endgroup$
    – cardinal
    Commented Dec 28, 2011 at 14:42
  • 2
    $\begingroup$ What kind of loss function are you interested in? You can always use general-purpose nonlinear minimization functions, but often there are better application-specific approaches. $\endgroup$
    – cardinal
    Commented Dec 28, 2011 at 14:45
  • $\begingroup$ But is there any other method to fit a logit function to a 0s & 1s response using a different loss function? $\endgroup$
    – user5497
    Commented Dec 28, 2011 at 14:47
  • $\begingroup$ there are several different functions. I get the "score" for the probabilities estimation according to these functions, so I want, the fitted function to be estimated according to those "scores" (and not just by using maximum likelihood $\endgroup$
    – user5497
    Commented Dec 28, 2011 at 14:49
  • 2
    $\begingroup$ @Adam: The logistic regression model is logically separate from the means by which one fits it. The model simply says that each observation is distributed as a Bernoulli independently of the other observations and with a probability that is a particular function of additional covariates. Choosing to fit such a model by maximum likelihood is an entirely separate matter. Choosing to implement a maximum-likelihood fit by IRLS is yet another, separate, matter. $\endgroup$
    – cardinal
    Commented Dec 28, 2011 at 23:13

1 Answer 1

1
$\begingroup$

Estimation is usually by maximizing the likelihood function. May be you can try probit instead of logit or other link function. I am not sure will this help you

$\endgroup$
3
  • $\begingroup$ Why can't I add comments to the original question? I am new to this forum $\endgroup$
    – vinux
    Commented Dec 28, 2011 at 14:53
  • 7
    $\begingroup$ There is no way to answer the question (other than pointing out that it contained a false assumption) without knowing the ultimate goal of the analysis. $\endgroup$ Commented Dec 28, 2011 at 14:55
  • $\begingroup$ @vinux: See the reputation section of the FAQ. (Total reputation of 50 is needed to comment everywhere.) $\endgroup$
    – cardinal
    Commented Dec 28, 2011 at 17:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.