Basic workflow of logistic regression is:
- Calculate log-odds of success (success means that your observation belongs
to the target class of interest)
- Convert log odds to percentage likelihood
- Apply some threshold to your percentage likelihood in order to apply a
predicted class label of "success" or "failure"
Your "a" term are known as the log-odds. These are the log of the odds that a sample belongs to a given class given that it possesses a set of predictors, to the odds that it does not belong to a certain class given its predictors. Think of it as the raw output from a logistic regression method in your native statistical software package (although sometimes you could also be given a probability, or even the class label directly, the log odds are driving all of these secondary outputs).
We cannot directly apply a linear regression to a classification problem since it violates the assumption that the response variables be continuous (i.e. exists between - infinity and + infinity). What we can say is that the odds in favor of an event occurring (in this case that your observation belongs to a class of interest), are positive and (usually) non-zero numbers. This means you can also calculate the log of those odds (the log odds from earlier). The beauty of the log odds is that they are continuous values that we can apply linear regression to.
The remainder of the work comes from finding the weights of your coefficients that maximize the probability of an observation belonging to the success class if it is indeed a success, and minimize the probability that are assigned to non-successes. This is controlled through the coefficient weights.
If you exponentiate the log-odds (your 'a' term) it is straightforward to calculate the probability.
Probability = (odds)/(odds+1)
Given that we have log-odds:
Probability = exp(a)/(exp(a)+1)
Divide out the exp(a) term and the probability statement reduces to your
sigma(a) = 1/ 1+exp(-a)