Why do the derivatives of a function lead towards the extremum of the function? Is there some theorem in mathematics that formalizes the idea that "for some function, at a given point, moving in the negative direction of the gradient leads you to some (local) extremum point"?  For example, in the gradient descent algorithm, what exactly justifies the general directions we follow when searching for the optimum point?
Recently, I read about Fermat's Theorem on Stationary Points, which says that if the point $(a,b)$ is a local extremum for some function $f$ then the derivative of $f$ at the point $(a,b)$ is 0.  Is there some extension of this theorem that states (the obvious idea) that moving in the negative direction of the gradient leads you to this (local) extremum point?  Or is this idea so basic and intuitive that no proof is required?
 A: Steepest ascent (descent) occurs in the direction (opposite direction) of the gradient
All theorems in mathematics are proved through detailed steps, even if the result is "obvious".  So as to your general question, no, it is not too obvious to require proof.  (When a result is an obvioius conseqeuence of some other result, such that it does not require proof, we call it a "corollary" rather than a "theorem".)  This particular result is proved through a combination of the multivariate chain rule and the geometric form for the dot product.
To get this result, consider a differentiable multivariate function $f:\mathbb{R}^n \rightarrow \mathbb{R}$ with gradient vector $\nabla f$, and suppose we choose a point $\mathbf{x} = (x_1,...,x_n)$ and a direction vector $\mathbf{u} = (u_1,...,u_n)$.  Applying the multivariate chain rule to the path function $\mathbf{x}(t) = \mathbf{x} + \mathbf{u} t$ we find that the directional derivative in the direction $\mathbf{u}$ from the point $\mathbf{x}$ is given by:
$$f_\mathbf{u}(\mathbf{x}) \equiv \frac{\partial f}{\partial \mathbf{u}}(\mathbf{x})
= \mathbf{u} \cdot \nabla f (\mathbf{x}).$$
Now, suppose we let $0 \leqslant \theta \leqslant \pi$ be the angle (in radians) between the direction vector $\mathbf{u}$ and the gradient vector $\nabla f(\mathbf{x})$.  Since $\mathbf{u}$ is a direction vector it has unit length, so we can apply the geometric meaning of the dot product to write the directional derivative as:
$$\begin{align}
f_\mathbf{u}(\mathbf{x}) 
&= \mathbf{u} \cdot \nabla f (\mathbf{x}) \\[6pt]
&= ||\mathbf{u}|| \ ||\nabla f (\mathbf{x})|| \cos \theta \\[6pt]
&= ||\nabla f (\mathbf{x})|| \cos \theta. \\[6pt]
\end{align}$$
The only thing in this equation that depends on the direction $\mathbf{u}$ is the angle $\theta$, so the directional derivative is affected by the direction only through this angle.  The steepest ascent occurs when $f_\mathbf{u}(\mathbf{x})$ is maximised for $\theta$, which occurs when the cosine of this angle is the most positive.  The steepest descent occurs when $f_\mathbf{u}(\mathbf{x})$ is minimised for $\theta$, which occurs when the cosine of this angle is the most negative.  Consequently, the angles giving steepest ascent/descent are as follows:$^\dagger$
$$\begin{matrix}
\text{Steepest ascent} & & \cos \theta = 1 \ \ \ & & \theta = 0 & & \text{(direction of gradient vector)} \\[6pt]
\text{Steepest descent} & & \cos \theta = -1 & & \theta = \pi & & \text{(opposite direction of gradient vector)} \\[6pt]
\end{matrix}$$
Another consequence of this result is that the gradient vector at a point $\mathbf{x}$ is always orthogonal to any tangent to the level curve of the function at that point.

$^\dagger$ This sets aside the special case where $\mathbf{x}$ is a critical point (i.e., when $\nabla f (\mathbf{x}) = \mathbf{0}$).  In this case the directional derivative is zero in all directions, so the steepest ascent/descent occurs (trivially) in any direction.
