I am learning measure theory and the concept $\sigma$-finite measure makes me a bit confused. Why do we need the $\sigma$-finite assumption in many important theorems?
I want to expand a little more on the answer given in the comments here. First, I think it's useful to give an intuitive explanation as to why it is so powerful. Then, I will discuss some examples where $\sigma$-finiteness is needed.
The main intuition is to think of $\sigma$-finite measures spaces as a collection of countably many finite measure spaces being glued together. Thus, when we prove results for $\sigma$-finite spaces we are essentially just finding a way to "string together" the result for these finite spaces. This should give you some idea of why it is so powerful since working within a finite measure space is a very strong assumption. Applying this intuition we see that $\sigma$-finite spaces are not really a major generalization away from finite spaces.
To give more insight into this condition I think it is useful to look at some results that can be achieved using $\sigma$-finite measures. The Radon-Nikodym Theorem is certainly one example and indeed it is a cornerstone of probability theory. However, I think there are a few other results that may help to shed some additional light on this condition.
First, from the perspective of probability theory, all non-zero $\sigma$-finite measures are equivalent to probability measures. That is, they agree on sets of measure zero. So there exists a measurable function $g: X\to (0,\infty)$ with $\int g d\mu = 1$ such that, $$P(A)=\int_A g d\mu$$
You can find a proof of this result here. The only modification is to use the function $g := f/\int f d\mu$ to create a probability measure.
Another very important theorem that typically requires $\sigma$-finite measure spaces is Fubini's Theorem. Specifically from the perspective of probability theory, this result allows us to work with joint distributions. For example, computing expectations of independent random variables.
Also, a very closely related concept here is the unique extension of measures. For non-$\sigma$-finite spaces, we get a failure of Fubini's theorem because the product measures will not extend uniquely without this condition (here is an example of this). In fact, notice that the Caratheodory Extension Theorem requires that the pre-measure be $\sigma$-finite for a unique extension.
A final reason for the importance of $\sigma$-finiteness comes from, perhaps, a slightly deeper result known as the Lebesgue Decomposition Theorem. This result tells us that given two $\sigma$-finite measures, $\mu$ and $\nu$, we can write,
$$\nu = \nu_a + \nu_s$$
where $\nu_a << \mu $ (is absolutely continuous w.r.t $\mu$) and $\nu_s \perp \mu$ (singular w.r.t. $\mu$). This theorem can actually be used to write a short proof of the Radon-Nikodym theorem (this is typically called the von Neumann proof).
Finally, we should note that there are generalizations of $\sigma$-finiteness that do allow us to develop Fubini's theorem and Radon-Nikodym. Still, we need some generalization of this condition. Furthermore, the equivalence of $\sigma$-finite measures to probability measures certainly is a strong motivation for using $\sigma$-finitenes when we develop these results in probability theory.