# Two Beginner Level Questions in ML [closed]

I am repeatedly surprised by how often these three things appear while any ML discussion is there:

1. Log-Likelihood: I understand the max likelihood principle, why log?
2. Softmax: Why softmax everywhere? Is it tied to log-likelihood in any way.
3. Sigmoid: Why sigmoid function only in NNs?

Please help me understand/direct me to resources which provided an intuitive + mathematical (rigorous) validations of these observations. Thanks a lot.

• Three quite distinct questions (your title is wrong), and some of the premises seem to be not wholly justified. You should consider posting separate questions, but beware - it's likely they're already answered. e.g. in relation to the first, consider this existing question or this one on math.SE (which doesn't really rely on the Gaussian part) and ... – Glen_b -Reinstate Monica Oct 3 '16 at 9:03
• ... this one on SO. So search carefully before editing to ask one question and before reposting the others – Glen_b -Reinstate Monica Oct 3 '16 at 9:07
• @Glen_b thanks for the links, but the truth is I tried looking for the questions separately, i wanted to see if there is smthng unifying these things. One derives or justifies others like taking log of softmax helps or smthng like that. – Sie Tw Oct 3 '16 at 9:44
• @Glen_b you didn't point to any solutions for question 2 and question 3. So, 2/3rd of the question is valid and plz remove the hold, not justified as I am more interested in knowing the linking b/w these 3 things. It is not too broad. Thanks. – Sie Tw Oct 3 '16 at 9:48
• No, the links for the first one were examples of what you can find with a simple search. It's not up to me to find them for you, it's up to you to search for them before posting your question.. – Glen_b -Reinstate Monica Oct 3 '16 at 11:06

2. Softmax is a way to transform distances from $(-\infty,\infty)$ interval to $(0,1)$ interval. This kind of mapping is useful mostly when you want to map distances into probability. For example consider you have a categorical distribution, like a variable which can take $k$ possible values, which can be represented as a vector of zeros with 1 on the selected category. However instead of binary variables you have a distance for each category, aka a vector of distances? How can you transform that into the desired space? One solution in softmax. Note that it is not the only one and there is not always a connection with log likelihood. Wikipedia page should give you clear details.