2
$\begingroup$

I am studying Machine Learning Probability perspective and I have a question with Algorithm 8.2, which is

$\mathbf{w} = 0_{D}$ $w_{0} = \log (\bar{y}/(1-\bar{y}))$

Repeat:
    $\eta_{i} = w_{0}+\mathbf{w}^{T} \mathbf{x}_{i}$
    $\mu_{i} = \text{sigm}(\eta_{i})$
    $s_{i} = \mu_{i}(1-\mu_{i})$
    $z_{i} = \eta_{i}+\frac{y_{i}-\mu_{i}}{s_{i}}$
    $\mathbf{S} = \text{diag}(s_{1:N})$
    $\mathbf{w} = (\mathbf{X}^{T}\mathbf{S}\mathbf{X})^{-1}\mathbf{X}^{T}\mathbf{S}\mathbf{z}$
until converged

I think $w_{0}$ means the coefficient of intercept, but why it is not updated in the loop?
It seems like $w_{0}$ is fixed like a constant?

$\endgroup$
3
$\begingroup$

I don't have the book but I believe this is a fitting algorithm for logistic regression.

In logistic regression, you can think the intercept like an estimate when the independent variable is zero. Of course, this is just the empirical estimate of log(p(Y=1)/p(Y=0). This is done in the second line:

$w_{0} = \log (\bar{y}/(1-\bar{y}))$

The intercept is also a constant because the log-odds don't change while the algorithm is running iterations.

More details: Intercept term in logistic regression

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.