We're using logistic regression to predict events probability. Logistic regression tries to minimize the residual variance (sum of squared residuals). However, in our specific problem we would like to use a different loss function (for the errors in the logistic regression). Does anyone knows any implementation (or how to implement) such change? (preferably in R or Python) Thanks!
$\begingroup$
$\endgroup$
10
-
6$\begingroup$ The typical approach to fitting logistic regression models is via maximum likelihood. This is not equivalent to minimizing the variance of the residuals. $\endgroup$– cardinalDec 28, 2011 at 14:42
-
2$\begingroup$ What kind of loss function are you interested in? You can always use general-purpose nonlinear minimization functions, but often there are better application-specific approaches. $\endgroup$– cardinalDec 28, 2011 at 14:45
-
$\begingroup$ But is there any other method to fit a logit function to a 0s & 1s response using a different loss function? $\endgroup$– user5497Dec 28, 2011 at 14:47
-
$\begingroup$ there are several different functions. I get the "score" for the probabilities estimation according to these functions, so I want, the fitted function to be estimated according to those "scores" (and not just by using maximum likelihood $\endgroup$– user5497Dec 28, 2011 at 14:49
-
2$\begingroup$ @Adam: The logistic regression model is logically separate from the means by which one fits it. The model simply says that each observation is distributed as a Bernoulli independently of the other observations and with a probability that is a particular function of additional covariates. Choosing to fit such a model by maximum likelihood is an entirely separate matter. Choosing to implement a maximum-likelihood fit by IRLS is yet another, separate, matter. $\endgroup$– cardinalDec 28, 2011 at 23:13
|
Show 5 more comments
1 Answer
$\begingroup$
$\endgroup$
3
Estimation is usually by maximizing the likelihood function. May be you can try probit instead of logit or other link function. I am not sure will this help you
-
$\begingroup$ Why can't I add comments to the original question? I am new to this forum $\endgroup$– vinuxDec 28, 2011 at 14:53
-
7$\begingroup$ There is no way to answer the question (other than pointing out that it contained a false assumption) without knowing the ultimate goal of the analysis. $\endgroup$ Dec 28, 2011 at 14:55
-
$\begingroup$ @vinux: See the reputation section of the FAQ. (Total reputation of 50 is needed to comment everywhere.) $\endgroup$– cardinalDec 28, 2011 at 17:29