Two Beginner Level Questions in ML [closed]

I am repeatedly surprised by how often these three things appear while any ML discussion is there:

1. Log-Likelihood: I understand the max likelihood principle, why log?
2. Softmax: Why softmax everywhere? Is it tied to log-likelihood in any way.
3. Sigmoid: Why sigmoid function only in NNs?

Please help me understand/direct me to resources which provided an intuitive + mathematical (rigorous) validations of these observations. Thanks a lot.

• Three quite distinct questions (your title is wrong), and some of the premises seem to be not wholly justified. You should consider posting separate questions, but beware - it's likely they're already answered. e.g. in relation to the first, consider this existing question or this one on math.SE (which doesn't really rely on the Gaussian part) and ... Commented Oct 3, 2016 at 9:03
• ... this one on SO. So search carefully before editing to ask one question and before reposting the others Commented Oct 3, 2016 at 9:07
• @Glen_b thanks for the links, but the truth is I tried looking for the questions separately, i wanted to see if there is smthng unifying these things. One derives or justifies others like taking log of softmax helps or smthng like that. Commented Oct 3, 2016 at 9:44
• @Glen_b you didn't point to any solutions for question 2 and question 3. So, 2/3rd of the question is valid and plz remove the hold, not justified as I am more interested in knowing the linking b/w these 3 things. It is not too broad. Thanks. Commented Oct 3, 2016 at 9:48
• No, the links for the first one were examples of what you can find with a simple search. It's not up to me to find them for you, it's up to you to search for them before posting your question.. Commented Oct 3, 2016 at 11:06

2. Softmax is a way to transform distances from $(-\infty,\infty)$ interval to $(0,1)$ interval. This kind of mapping is useful mostly when you want to map distances into probability. For example consider you have a categorical distribution, like a variable which can take $k$ possible values, which can be represented as a vector of zeros with 1 on the selected category. However instead of binary variables you have a distance for each category, aka a vector of distances? How can you transform that into the desired space? One solution in softmax. Note that it is not the only one and there is not always a connection with log likelihood. Wikipedia page should give you clear details.