# theoretical basis for logistic regression

Sorry if this is obvious but why does it seem that some papers/studies (usually in economics discipline) use utility theory as a basis for discrete choice models while others (often social science) do not? If what is being modelled is a binary choice, is a specification of the reasons why this choice is being made (as in, to maximise utility) and an accompanying utility model necessary? It just seems some use it and some don't, even when what is being modelled is essentially the same thing. Thanks

There is no theoretical basis for logistic regression (in general as a choice vs. another model). Two things are arbitrary:

• summing the influences of each variables, each influence being proportional to the variable (linear predictor)
• the sigmoid link (logit)

The first assumption is similar to linear regression: a simple model that is very useful and often matches observations sufficiently well to make something of it.

The second assumption can't be justified either. It is similar to the assumption of normality of the noise in linear regression. Interestingly many other link functions produce very similar results: Difference between logit and probit models.

It is however interesting that logistic regression is equivalent to maximum entropy (in the case of binary/multinomial outcomes and independent observations), and that maximum entropy was stated as a principle by Jaynes in the 50s. I think people realized the two are equivalent much later (early 2000s as far as I known).

• Sigmoid/softmax is how you convert logarithmic probabilities to probabilities. Logarithmic probabilities are convenient because bayes' theorem simplifies to addition. Summing the influences of each variable amounts to assuming their independence, which is a weaker assumption than normally distributed noise. So IMO logistic regression has a pretty string theoretical basis. – CodesInChaos Dec 28 '17 at 19:57
• "The second assumption can't be justified either. It is similar to the assumption of normality of the noise in linear regression." The assumption of normality in LR is not completely arbitrary and can often be justified thinking of the additive noise as the result of a large number of independent small sources of error. By the CLT, their combined effect approaches a normal distribution. – Luca Citi Dec 28 '17 at 23:52

No, it isn't necessary. Economists like to get elbow-deep in mathematical hypotheses of how people make decisions, hence their frequent invocation of utility theory. But logistic regression can be justified in statistical terms without reference to utility, using the idea that a unit change in a predictor relates to an additive change in the log odds of a response.

• @B.Lari You're welcome. If I answered your question to your satisfaction, you can accept my answer by clicking the check mark under the voting arrows. – Kodiologist Jan 16 '18 at 17:14

You get an impression that economists do it because that's what they are forced to write in order to get published in microeconomics. Pure empirical studies are hard to publish.

However, this is changing, and not all economists do this. For instance, take a look at this work: "Analyzing the Risk of Mortgage Default." They use multinomial logit, and there's no utility function mentioned anywhere in the paper. And this is not even macroeconomics, where they don't feel pressured to shove the utility function in every paper

Other people have answered your question, let me explain a bit more the philosophy behind the different justfications for logit models.

The utility model used in economics is based on the grand idea to link general preference orderings over outcomes with the ordering of real numbers. Less abstractly, what economists have tried to do is to show when any preference over some possible outcomes can be represented by functions that give a "maximum choice", and when this is not possible.

This fits very natural with logistic regression when there are only two choices, 0 and 1, and also very well with multinomial models where there are more choices. Given distributional assumptions, the logistic regression therefore "arises" naturally out of a microfounded and very general model of human behavior. This is nice for economists, because many results they are after abstractly require the existence of such preferences to make sense more than just heuristically. The same is true for other social sciences that rely on choice, but their focus is often different.

One can posit a discrete choice model either as a utility model, or a latent variable model. The latent variable model (where $y=1$ if some $y*>t$) is also basically a choice model, only that it does not specify why this decision rule comes about.

Sometimes we are not interested in modeling this why. For example, we may simply not care, because some otherwise stable but complicated mechanism is behind it. It may also be the case that there is no actual entity making a decision, it is in a sense a purely statistical affair. It would then be rather contrived to think about some hypothetic preference orderings by some non-entity.

So to answer your question: The utility model is not necessary at all. It depends on your research question. Is there an entity making a decision? If so, are you trying to learn something about this decisionmaking? If yes, then all approaches will sooner or later lead to a utility model, simply because you need to find stable or logical "preferences" in your research.

In other applications, utility is not necessary at all (especially outside of social sciences this may be the case, say a mechanical model) and it would be unnecessary and even harmful to argue with the utility model.

I'm not an economist nor do I know much utility theory, but I actually think there is some theoretical justification for logistic regression - at least at a high level. In real life, aren't decisions closer to existing on a scale like 0-100% rather than 0/1 binary? The ability to get 'probabilities' out of a logistic model makes it more attractive than some other classification methods. Of course, the fact that linearly separability frustrates logistic regression poses a problem for certain classes of theoretical justification.