# How is the minimum logarithmic loss calculated when initializing the XGBoost algorithm?

Suppose there are $$5$$ sample units, $$2$$ of which carry the feature $$y=1$$ to be predicted and three of which carry the feature $$y=0$$. So, $$2$$ are positive. The XGBoost algorithm initializes with

$$\hat\theta_0=argmin_\theta\sum_{i=1}^nL(y_i,\theta)$$,

where $$L(y_i,\theta)$$ is the logarithmic loss $$L(y_i,\theta)=\frac{1}{n}\sum_{i=1}^ny_i\log(p_i)+(1-y_i)\log(1-p_i)$$,

where $$p_i=\frac{1}{1+e^{-x_i}}.$$

Now, I am struggling a bit with how to calculate the initial leaf weights. These should be represented by $$x_i$$ in the equation as far as I understand? From a common sense perspective, we should start with $$\theta_0=0.4$$ (which is the sample mean and maximum likelihood estimator of a bernoulli distribution). But how can I solve the minimization problem so that I come to this result?

$$L(y_i,\theta)$$ solves to $$2*log(p_i)+3*log(1-p_i)$$. But, this should be solved by $$\theta_0=x=1$$ which seems to be an odd initial value to start the XGBoost algorithm. Or is XGBoost actually starting with a prediction of 1 for every sample unit?

Okey, so I have an answer to the question: I have to differentiate with respect to $$\theta=p_i$$ instead of $$x_i$$. Then, $$2*log(p_i)+3*log(1-p_i)$$ is minimized by $$\theta=p_i=0.4$$.

• Just to be clear: there are 5 samples and 2 of them are positive? – usεr11852 says Reinstate Monic Feb 25 '19 at 23:27
• Congratulations on answering your own question and good on you for sharing the answer! (I had on the back of my head to write a quick answer about it this weekend) You might want write a short concise answer about it so people can potentially upvote it (I would). – usεr11852 says Reinstate Monic Mar 1 '19 at 0:30