Derivative equal to zero in PCA Source: presentation
In PCA, we want to maximize variance $var(\alpha'_kX)$, i.e. $\alpha'_k \Sigma \alpha_k$ with respect to $\alpha' \alpha = 1$, where $\Sigma = X'X$. After introducing Lagrange multipliers, we set $\alpha_k$ derivative equal to zero in order to maximize variance, i.e. $\frac{d}{d \alpha_k} (\alpha_k' \Sigma \alpha_k - \lambda_k (\alpha_k' \alpha_k - 1)) = 0$.
How does above derivative yield maximal variance?
 A: The variance of $a^T X$ is given by $a^T\Sigma a$ so we maximize the Lagrangian $g(a) = a^T\Sigma a + \lambda(1 - a^Ta)$ and since this is differentiable and we're optimizing it over all of $\mathbb R^p$, the extrema will necessarily be at zeros of the derivative. This means
$$
\frac{\partial g}{\partial a} = 2\Sigma a -  2\lambda a \stackrel{\text{set}}=0
$$
is a necessary but not sufficient condition for the maximum. In other words, the maximum must be for a point $v$ such that
$$
\Sigma v = \lambda v
$$
for some $\lambda$. But there are many such points and this first order condition by itself does not tell us which of them to pick; this alone only tells us that the argmax will be a unit eigenvector of $\Sigma$.
Now as a second step we can use this to find the actual maximum. For any eigenvector $a$ of $\lambda$ we'll have 
$$
g(a) = a^T\Sigma a + \lambda(1-a^Ta) = \lambda
$$
so now that we just need to optimize over the eigenvectors, all we need to do is pick a unit eigenvector of $\Sigma$ with the largest eigenvalue. But this is done using the extra information we know about what it means to be an eigenvector and isn't just from directly solving $\frac{\partial g}{\partial a}=0$, since for example the non-zero $a$ that minimizes the variance is also a solution to $\frac{\partial g}{\partial a}$ (this would be an eigenvector of the smallest eigenvalue).
