Validity of maximising log-likelihood for maximum likelihood estimation For reasons owing to mathematical convenience, when finding MLEs (maximum likelihood estimates), it is often the log-likelihood function---as opposed to the standard likelihood function---which is maximised.
From what I've gathered, this approach is deemed valid as a result of the monotonically increasing nature of the (natural) logarithm function.
My understanding of a monotonically increasing function is: for all $x$ and $y$ (defined on a subset of the reals), if $x \leq y$ then $f(x) \leq f(y)$.
This, however, does not appear to be the case for all log-likelihood functions; for example: for the log-likelihood $\text{Gamma}(3, 5)$ function, if $x = 0.15$ and $y = 0.46$, then $f(x) = -0.59$ and $f(y) = -1.19$.
Clearly, I've misunderstood this concept. Fundamentally, I guess I'm asking can somebody (preferably mathematically) demonstrate why:
$$ \hat{\theta} = \text{argmax} \text{ } L(\theta) = \text{argmax} \text{ } \text{log} \text{ } L(\theta) $$
where $\hat{\theta}$ is the MLE for a given likelihood function.
 A: This is a direct consequence of properties of monotone (increasing) transformation, and the logarithm is monotone increasing. If there exist a value of $\theta$ that maximizes the likelihood function, that same value of $\theta$ will maximize the log likelihood function.  The later is often preferred because it has better numerical properties, so is easier to maximize in practice. That is not the only reason, the log likelihood function arises also much in theory.
EDIT The answer by @David Grenier: shows this by using calculus. But calculus is not necessary! Let $f$ be a monotonous increasing function, like $\log$. This means that for all $x,y$, if $x\le y$ then $f(x) \le f(y)$. Let $\hat\theta$ be an MLE, so that for all $\theta$, $L(\theta)\le L(\hat\theta)$ where $L$ is the likelihood function.
Applying $f$ to both sides we can conclude $f(L(\theta))\le f(L(\hat\theta))$. Now, if $f$ (like $\log$) is strictly increasing, it has an inverse function, so the above argument can be reversed.
A: Here's a simple proof that may help from a mathematical perspective. It's mostly just basic calculus.
Let $L(\theta)$ be a continuous function such that $L(\theta) > 0$ for some interval on the real number line (the support). Let $L(\theta)$ be maximized at $\hat{\theta}$.
If $L(\theta)$ is maximized at $\hat{\theta}$ then by the first and second derivative tests, $L^{\prime}(\hat{\theta}) = 0$ and $L^{\prime \prime}(\hat{\theta}) < 0$. Further as defined above we know $L(\hat{\theta}) > 0$. This gives us: $L(\hat{\theta}) > 0$; $L^{\prime}(\hat{\theta}) = 0$; $L^{\prime \prime}(\hat{\theta}) < 0$.
Let $\ell(\theta) = \ln[L(\theta)]$. Then $\ell(\theta) = (f \circ g)(\theta)$ where $f = \ln(x)$ and $g = L(x)$
If $\ell(\theta) = (f \circ g)(\theta)$ then by the chain rule $\ell^{\prime}(\theta) = (f' \circ g)g' = \frac{1}{L(\theta)} \cdot L^{\prime}(\theta) = \frac{L^{\prime}(\theta)}{L(\theta)}$
If $\ell^{\prime}(\theta) = \frac{L^{\prime}(\theta)}{L(\theta)}$ then by the quotient rule $\ell^{\prime \prime} = \frac{L^{''}L - L'L'}{L^2}$
If $\ell^{\prime}(\theta) = \frac{L^{\prime}(\theta)}{L(\theta)}$ and $L^{\prime}(\hat{\theta}) = 0$, then $\ell^{\prime}(\hat{\theta}) = \frac{0}{L(\hat{\theta})} = 0$ and $\ell(\hat{\theta})$ is a critical point.
If $\ell^{\prime \prime} = \frac{L^{''}L - L'L'}{L^2}$ and $L^{\prime}(\hat{\theta}) = 0$ then $\ell^{\prime \prime}(\hat{\theta}) = \frac{L^{''}(\hat{\theta})L(\hat{\theta})}{\left[L(\hat{\theta})\right]^2}$. Further, if $L(\hat{\theta}) > 0$ and $L^{\prime \prime}(\hat{\theta}) < 0$, then $\frac{L^{''}(\hat{\theta})L(\hat{\theta})}{\left[L(\hat{\theta})\right]^2} < 0$ and $\ell^{\prime \prime}(\hat{\theta}) < 0$
If $\ell^{\prime}(\hat{\theta}) = 0$ and $\ell^{\prime \prime}(\hat{\theta}) < 0$ then $\ell(\hat{\theta})$ is a minimum on the support of $L$.
Therefore if $L(\theta)$ is maximized at $\hat{\theta}$ then $\ln[L(\theta)]$ will also be maximized at $\hat{\theta}$.
