# Why typically minimizing a cost instead of maximizing a reward?

I understand that, for example, maximizing the log-likelihood is equivalent to minimizing the negative log-likelihood. It is indeed a simple change, but still an extra step taken (it seems) for the unique purpose of designing a loss function that will be minimized instead of maximized.

I wonder why this has become the standard in Machine Learning?

• Is there any numerical consideration that favors function minimization instead of maximization?
• Why has gradient descent become such a universal standard? (I have never seen a Deep Learning paper in which they use gradient ascent to directly maximize the likelihood)

Disclaimer :

I came across many similar questions, but none of which that have been truly answered. People typically just explain how both approaches are equivalent, or explain why we use the logarithm for numerical stability, but without explaining why minimization is favored over maximization. (See those two questions : 1, 2)

• The distinction between minimization of an objective and maximization of the negative of the objective function is so trivial that nobody is concerned about it. Many authors and software implementations have settled on the minimization convention, that's all. One can scarcely characterize this as an "extra step"--it's no different than having to pay attention to any other convention, such as what order to pass parameters to a function or whether to record lengths in feet or meters. – whuber Oct 17 '17 at 19:38
• I've heard that although they are mathematically identical, minimization is preferred due to the nature of the numerical computations the computer has to do. If so, it may be that someone on, say, Computational Science, Computer Science, or Theoretical Computer Science could provide a good explanation. – gung - Reinstate Monica Oct 17 '17 at 20:07
• @gung The only difference in most digital computers between a negative and positive (floating point) value is the setting of the sign bit. There's no difference in precision or in performance of the numerical operations. – whuber Oct 17 '17 at 20:28
• Simply a convention in optimization - see 1, 2, 3 ... ctd – Glen_b -Reinstate Monica Oct 17 '17 at 21:40
• ctd... Also see this, which makes it clear that it is effectively arbitrary. $\:$ It makes sense to settle on a single convention, so that people that work on the mathematics of optimization algorithms (and the people who code them) can focus on the essentials instead of constantly converting back and forth between different optimization conventions (these authors minimize; those maximize). Ultimately it also makes it easier on users. – Glen_b -Reinstate Monica Oct 17 '17 at 21:40