# Logistic regression and maximum entropy

I have read (e.g. here) that a (multinomial) logistic regressor corresponds to a maximum entropy classifier.

My question is, how does one end up with the formula for logistic regression starting with the maximum entropy principle?

• One way is by considering the dual convex program. – cardinal Nov 22 '13 at 17:24
• Thanks @cardinal - Do you know of any sources that I can look up to learn more about the dual convex derivation? – Josh Nov 22 '13 at 18:51
• Hi Josh. I'm not entirely sure where to best direct you off-hand. You might look in Boyd & Vandenberghe (available for free online) to start. I will hunt around here in a bit and see if I can give you a more definitive pointer. Are you familiar with convex programming? – cardinal Nov 22 '13 at 19:20
• Thanks @cardinal. I am familiar with convex programming and duality. I guess what I am looking for is: (1) the problem formulation (2) the formulation of maximum entropy (I presume as constraints), and somehow a proof that logistic regressor emerges as the optimal solution. – Josh Nov 22 '13 at 19:24
• See win-vector.com/dfiles/LogisticRegressionMaxEnt.pdf for an explanation – kjetil b halvorsen Apr 12 '18 at 8:33