Understanding the Logistic Regression and likelihood How does the parameter estimation/Training of logistic regression really work? I'll try to put what I've got so far. 


*

*The output is y the output of the logistic function in form of a probability  depending on the value of x :
$$P(y=1|x)={1\over1+e^{-\omega^Tx}}\equiv\sigma(\omega^Tx)$$
$$P(y=0|x)=1-P(y=1|x)=1-{1\over1+e^{-\omega^Tx}}$$

*For one dimension the so called Odds is defined as follows:
$${{p(y=1|x)}\over{1-p(y=1|x)}}={{p(y=1|x)}\over{p(y=0|x)}}=e^{\omega_0+\omega_1x}$$

*Now adding the log function to get the W_0 and W_1 in linear form:
$$Logit(y)=log({{p(y=1|x)}\over{1-p(y=1|x)}})=\omega_0+\omega_1x$$

*Now to the problem part  Using the likelihood (Big X is y ) 
$$L(X|P)=\prod^N_{i=1,y_i=1}P(x_i)\prod^N_{i=1,y_i=0}(1-P(x_i))$$
Can any one tell why we're considering the probability of y=1 twice ?
since :
$$P(y=0|x)=1-P(y=1|x)$$
and how get the values of ω from it?
 A: Your likelihood function (4) consists of two parts: the product of the probability of success for only those people in your sample who experienced a success, and the product of the probability of failure for only those people in your sample who experienced a failure. Given that each individual experiences either a success or a failure, but not both, the probability will appear for each individual only once. That is what the $, y_i=1$ and $,y_i=0$ mean at the bottom of the product signs.
The coefficients are included in the likelihood function by substituting (1) into (4). That way the likelihood function becomes a function of $\omega$. The point of maximum likelihood is to find the $\omega$ that will maximize the likelihood. 
A: Assume in general that you decided to take a model of the form
$$P(y=1|X=x) = h(x;\Theta)$$
for some parameter $\Theta$. Then you simply write down the likelihood for it, i.e.
$$L(\Theta) = \prod_{i \in \{1, ..., N\}, y_i = 1} P(y=1|x=x;\Theta) \cdot \prod_{i \in \{1, ..., N\}, y_i = 0} P(y=0|x=x;\Theta)$$
which is the same as
$$L(\Theta) = \prod_{i \in \{1, ..., N\}, y_i = 1} P(y=1|x=x;\Theta) \cdot \prod_{i \in \{1, ..., N\}, y_i = 0} (1-P(y=1|x=x;\Theta))$$
Now you have decided to 'assume' (model)
$$P(y=1|X=x) = \sigma(\Theta_0 + \Theta_1 x)$$
where $$\sigma(z) = 1/(1+e^{-z})$$
so you just compute the formula for the likelihood and do some kind of optimization algorithm in order to find the $\text{argmax}_\Theta L(\Theta)$, for example, newtons method or any other gradient based method.
Notce that sometimes, people say that when they are doing logistic regression they do not maximize a likelihood (as we/you did above) but rather they minimize a loss function 
$$l(\Theta) = -\sum_{i=1}^N{y_i\log(P(Y_i=1|X=x;\Theta)) + (1-y_i)\log(P(Y_i=0|X=x;\Theta))}$$
but notice that $-\log(L(\Theta)) = l(\Theta)$.
This is a general pattern in Machine Learning: The practical side (minimizing loss functions that measure how 'wrong' a heuristic model is) is in fact equal to the 'theoretical side' (modelling explicitly with the $P$-symbol, maximizing statistical quantities like likelihoods) and in fact, many models that do not look like probabilistic ones (SVMs for example) can be reunderstood in a probabilistic context and are in fact maximizations of likelihoods.
