Logistic regression can be described as a linear combination
$$ \eta = \beta_0 + \beta_1 X_1 + ... + \beta_k X_k $$
that is passed through the link function $g$:
$$ g(E(Y)) = \eta $$
where the link function is a logit function
$$ E(Y|X,\beta) = p = \text{logit}^{-1}( \eta ) $$
where $Y$ take only values in $\{0,1\}$ and inverse logit functions transforms linear combination $\eta$ to this range. This is where classical logistic regression ends.
However if you recall that $E(Y) = P(Y = 1)$ for variables that take only values in $\{0,1\}$, than $E(Y | X,\beta)$ can be considered as $P(Y = 1 | X,\beta)$. In this case, the logit function output could be thought as conditional probability of "success", i.e. $P(Y=1|X,\beta)$. Bernoulli distribution is a distribution that describes probability of observing binary outcome, with some $p$ parameter, so we can describe $Y$ as
$$ y_i \sim \text{Bernoulli}(p) $$
So with logistic regression we look for some parameters $\beta$ that togeder with independent variables $X$ form a linear combination $\eta$. In classical regression $E(Y|X,\beta) = \eta$ (we assume link function to be identity function), however to model $Y$ that takes values in $\{0,1\}$ we need to transform $\eta$ so to fit in $[0,1]$ range.
Now, to estimate logistic regression in Bayesian way you pick up some priors for $\beta_i$ parameters as with linear regression (see Kruschke et al, 2012), then use logit function to transform the linear combination $\eta$, so to use its output as a $p$ parameter of Bernoulli distribution that describes your $Y$ variable. So, yes, you actually use the equation and the logit link function the same way as in frequentionist case, and the rest works (e.g. choosing priors) like with estimating linear regression the Bayesian way.
The simple approach for choosing priors is to choose Normal distributions (but you can also use other distributions, e.g. $t$- or Laplace distribution for more robust model) for $\beta_i$'s with parameters $\mu_i$ and $\sigma_i^2$ that are preset or taken from hierarchical priors. Now, having the model definition you can use software such as JAGS to perform Markov Chain Monte Carlo simulation for you to estimate the model. Below I post JAGS code for simple logistic model (check here for more examples).
model {
# setting up priors
a ~ dnorm(0, .0001)
b ~ dnorm(0, .0001)
for (i in 1:N) {
# passing the linear combination through logit function
logit(p[i]) <- a + b * x[i]
# likelihood function
y[i] ~ dbern(p[i])
}
}
As you can see, the code directly translates to model definition. What the software does is it draws some values from Normal priors for a and b, then it uses those values to estimate p and finally, uses likelihood function to assess how likely is your data given those parameters (this is when you use Bayes theorem, see here for more detailed description).
The basic logistic regression model can be extended to model the dependency between the predictors using a hierarchical model (including hyperpriors). In this case you can draw $\beta_i$'s from Multivariate Normal distribution that enables us to include information about covariance $\boldsymbol{\Sigma}$ between independent variables
$$ \begin{pmatrix} \beta_0 \\ \beta_1 \\ \vdots \\ \beta_k \end{pmatrix} \sim \mathrm{MVN} \left(
\begin{bmatrix} \mu_0 \\ \mu_1 \\ \vdots \\ \mu_k \end{bmatrix},
\begin{bmatrix} \sigma^2_0 & \sigma_{0,1} & \ldots & \sigma_{0,k} \\
\sigma_{1,0} & \sigma^2_1 & \ldots &\sigma_{1,k} \\
\vdots & \vdots & \ddots & \vdots \\
\sigma_{k,0} & \sigma_{k,1} & \ldots & \sigma^2_k
\end{bmatrix}
\right)$$
...but this is going into details, so let's stop right here.
The "Bayesian" part in here is choosing priors, using Bayes theorem and defining model in probabilistic terms. See here for definition of "Bayesian model" and here for some general intuition on Bayesian approach. What you can also notice is that defining models is pretty straightforward and flexible with this approach.
Kruschke, J. K., Aguinis, H., & Joo, H. (2012). The time has come: Bayesian methods for data analysis in the organizational sciences. Organizational Research Methods, 15(4), 722-752.
Gelman, A., Jakulin, A., Pittau, G.M., and Su, Y.-S. (2008). A weakly informative default prior distribution for logistic and other regression models. The Annals of Applied Statistics, 2(4), 1360–1383.
