# Logistic regression produces well calibrated models. Is that true for neural nets trained in batches?

This is an earlier discussion about LR producing well calibrated models:

Some people equate neural net based prediction models (even deep NN or deep+sparse NN) to be equivalent to logistic regression. We train them with adagrad (or some other methods) but always update weights by optimizing cost function on limited batch.

Q1: Is it true that Neural Nets have properties of logistic regression?

Q2: When we train with batches, weights that were calculated in the first batch and produce 'well calibrated' output, probably will be updated quite differently when processing subsequent batch. Intuitively, I think after a few updates this 'well-calibrated' property wouldn't hold (and especially for sparse neural nets where some embedding table is used). Is this correct?

• A neural network with sigmoid activation and without hidden layers is a logistic regression. However, a simple counterexample shows that the logic used in the answer you linked does not apply to neural networks: Training a model on data with a low incidence rate without supplying weights often causes the neural network to predict all observations to have the same class. In such a case the sum of predicted class probabilities obviously does not equal the sum of the outcome. May 3, 2019 at 23:19
• @FransRodenburg is it though? What about probit regression, or cloglog? All of those link functions also have the shape of a sigmoid (as do most CDFs), but why specifically logistic? Or was it because the OP asked about logistic?
– runr
May 22, 2019 at 8:54
• @Nutle In neural networks, sigmoid almost always refers to the logistic function $\frac{1}{1 + e^{-x}}$, as opposed to other (possibly S-shaped) activation functions like softmax, tanh, ReLU, etc. May 22, 2019 at 11:54
• @FransRodenburg Thanks. You're right, my mistake. It's interesting how in neural networks a specific case of a specific sigmoid (logit) function is called as the whole family of curves, but that's off topic.
– runr
May 22, 2019 at 12:29