Linked Questions
12 questions linked to/from Why do we minimize the negative likelihood if it is equivalent to maximization of the likelihood?
5
votes
1
answer
6k
views
Maximizing likelihood vs. minimizing cost [duplicate]
I keep coming across two different kinds of optimization:
Cases where you maximize the likelihood of the data directly (for example CRF learning, or EM).
Cases where you minimize some cost function (...
- 163
4
votes
3
answers
4k
views
why minimize loss function instead of maximizing reward function? [duplicate]
Why is the "de-facto" in statistics to minimize the sum of squared errors cost function instead of maximizing some reward function like the likelihood function?
- 665
6
votes
0
answers
731
views
Why do we minimise a cost function instead of maximising an equivalent? [duplicate]
I don't really understand why we minimise a cost function for gradient descent. Why don't we try to have something like a gradient 'climb', where we maximise some function?
Is it due to convention, or ...
- 61
98
votes
7
answers
44k
views
Why to optimize max log probability instead of probability
In most machine learning tasks where you can formulate some probability $p$ which should be maximised, we would actually optimize the log probability $\log p$ instead of the probability for some ...
- 1,265
68
votes
3
answers
98k
views
Cross-Entropy or Log Likelihood in Output layer
I read this page:
http://neuralnetworksanddeeplearning.com/chap3.html
and it said that sigmoid output layer with cross-entropy is quite similiar with softmax output layer with log-likelihood.
what ...
- 1,031
13
votes
2
answers
7k
views
Is binary logistic regression a special case of multinomial logistic regression when the outcome has 2 levels?
Is it correct to say that binary logistic regression is a special case of multinomial logistic regression when the outcome has 2 levels?
- 1,507
3
votes
6
answers
4k
views
Why typically minimizing a cost instead of maximizing a reward?
I understand that, for example, maximizing the log-likelihood is equivalent to minimizing the negative log-likelihood. It is indeed a simple change, but still an extra step taken (it seems) for the ...
- 507
9
votes
1
answer
5k
views
Cross entropy vs KL divergence: What's minimized directly in practice?
My understanding is that in ML one can establish a connection between these quantities using the following line of reasoning:
Assuming we plan to use ML to make decisions, we choose to minimize our ...
- 4,598
4
votes
1
answer
3k
views
Why use KL-Divergence as loss over MLE?
I have came across this statement several time now
Maximizing likelihood is equivalent to minimizing KL-Divergence
(Sources: Kullback–Leibler divergence and Maximum likelihood as minimizing the ...
- 237
2
votes
1
answer
533
views
logarithm in loglikelihood
I think I understand likelihood, and also log-likelihood as well. After reading about log-likelihood in various sources, I thought that the purpose of taking the logarithm of likelihood was all about ...
- 163
1
vote
1
answer
129
views
Two Beginner Level Questions in ML [closed]
I am repeatedly surprised by how often these three things appear while any ML discussion is there:
Log-Likelihood: I understand the max likelihood principle, why log?
Softmax: Why softmax everywhere? ...
- 439
2
votes
0
answers
54
views
derivative of Logistic Regression (sigmoid) [closed]
I am having difficulty figuring out, why I get different answer from the professor. we are tasked with finding the deriative of the logistic regression cost function with the sigmoid function:
$$ L(w│...
- 21