Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

What does the ridge parameter change in a Logistic Regression classifier as for example implemented in Weka Logistic classifier "Parameter -R ridge". The paper describing the underlying theory: Ridge Estimators in Logistic Regression. Any help is much appreciated!

share|improve this question
    
Please use the correct terminology. The logistic regression model is not a classifier. It is a direct probability model. –  Frank Harrell Oct 4 at 2:35

2 Answers 2

Presumably, this is done by applying the ridge estimator to the iteratively reweighted least squares (IRWLS) method.

share|improve this answer

Penalized maximum likelihood estimation with a quadratic penalty function is often called ridge logistic regression. The penalty is not applied to the intercept. The higher the value of the penalty parameter (aka ridge parameter) the closer to zero are the penalized maximum likelihood estimates. The optimum penalty can often be chosen using "effective AIC" as shown in http://biostat.mc.vanderbilt.edu/wiki/pub/Main/FHHandouts/iscb98.pdf.

Once the penalized log-likelihood is formed, maximizing the penalized log likelihood can be done quite easily using standard methods such as Newton-Raphson with step-halving.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.