0
$\begingroup$

This question already has an answer here:

Please excuse me if my wording is bad. I am trying to get a hold of the concepts. To minimize the loss function for real values, we can use SVD to solve the following:

$\arg \min_{A, B \in \mathcal{B}} \sum_{i,j} (A[i,] \times B[,j] - X[i,j])^2$

To minimize a logistic loss function, however, I am not sure how to handle it:

$\arg \min_{b} \sum_{i=1}^n \log(1+\exp(b^T a_i)) - x_i b^T a_i$

Since it is not the same form as above due to the exponential, we can't just use the solution analogously. I have tried taking the derivative and setting it to 0, but I am just stuck now. Any ideas?

EDIT: Please note that I am not necessarily looking for an analytical solution. A numerical one is fine. For instance, I have heard of alternating minimization, but how do I use it in this context?

$\endgroup$

marked as duplicate by Sycorax, kjetil b halvorsen, gung, John, mdewey Nov 21 '16 at 9:15

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 1
    $\begingroup$ For the exact reasons you outline this can't be solved using an analytical expression. The workhorse for fitting models of this type is Newton's method. $\endgroup$ – Sycorax Nov 20 '16 at 17:26
  • $\begingroup$ @Firebug it looks similar to what Sycorax is suggesting. I have updated my question to specify that I am not looking for an analytical solution. $\endgroup$ – FullStack Nov 20 '16 at 17:48
  • $\begingroup$ @FullStack There's a discussion of how to fit logistic regression in Elements of Statistical Learning which is available for free from the authors. statweb.stanford.edu/~tibs/ElemStatLearn/printings/… $\endgroup$ – Sycorax Nov 20 '16 at 18:32