Logistic regression link function from optimization perspective

I learned the logistic regression from optimization context instead of statistics context.

Logistic regression is trying to minimize logistic loss. I know there are two forms, but they are identical. Essentially from probability perspective, we are doing max likelihood on Bernoulli probability models.

Why there are two different logistic loss formulation / notations?

Now how the other link function play different rows here? In addition to logit link we have probit link

Difference between logit and probit models

Are we essentially changing logistic loss to something else? Why I never heard of "probit loss"?

• I tried to give an answer, as I come from a similar perspective! Note that you have a typo in the question title: missing "i" in link – GeoMatt22 Sep 19 '16 at 20:19
• @GeoMatt22 thanks for your answer and suggestions. typo fixed! – Haitao Du Sep 19 '16 at 20:20
• What is the question here exactly? Are you asking if there is a way to formulate bernoulli regression with a probit link as a loss of the form $\log(1+ \exp(−y \hat{y}))$, as per the question you linked? – Andrew M Sep 19 '16 at 20:48
• @AndrewM I was trying to ask what's the objective function for other link functions. and if there are any intuitive explanations for other objective functions. – Haitao Du Sep 20 '16 at 1:17

As I understand it*, link functions are associated with Generalized linear modeling (GLM). The link function is used to relate the (conditional) expected value of the dependent variable $y$ to a linear predictor constructed from the independent variables $x$, i.e. $$g[\langle y\mid x\rangle]=L[x,\theta]$$ where $g[\,]$ is the link function and $L[\,]$ is the linear predictor, parameterized by $\theta$, which is to be estimated by MLE, assuming i.i.d. data $y$. The link function is relatively unconstrained, but to be admissable it must have an appropriate domain and range, and it must be invertible.
For the case of Bernoulli distributed data $y\sim\mathrm{Bern}[p]$, the left hand side becomes $g[\,p[x]\,]$, where $p[x]=\langle y\mid x\rangle$ is the conditional mean.