Differencing is mainly appropriate when the time series $\{x_t\}$ contains one or more unit roots, i.e. is integrated of order $d=1,2,...$. In case of a single unit root, $\{x_t\}$ is I(1); it is a cumulative sum of an I(0) time series; $x_t=\sum_{\tau=1}^t u_\tau$ where $\{u_\tau\}$ is I(0). An example of an I(1) series could be a logarithmic stock price where the logarithmic returns are approximately I(0). Taking the first difference of an I(1) series allows us to work with the I(0) series, and that is convenient as we have all kinds of useful statistical results derived for stationary series*.

In case of two unit roots, the first-differenced time series $\{\Delta x_t\}=\{x_t- x_{t-1}\}$ is a cumulative sum of an I(0) time series. In other words, the original series is a "twice-cumulative sum" of an I(0) time series: $x_t=\sum_{\tau=1}^t\left(\sum_{s=1}^\tau v_s\right)$ where $\{v_s\}$ is I(0). Some economists consider the consumer price index to be approximated by I(2) which implies that the price increments are I(1). Indeed, price increments show some persistence, but how good of an approximation I(1) is to them is debatable.

I am not aware of any I(3) processes in the real world.

*An I(0) time series is quite similar to a stationary one, though there are some subtle differences.

Differencing is mainly appropriate when the time series $\{x_t\}$ contains one or more unit roots, i.e. is integrated of order $d=1,2,...$.

I(1): In case of a single unit root, $\{x_t\}$ is I(1); it is a cumulative sum of an I(0) time series, $x_t=\sum_{\tau=1}^t u_\tau$ where $\{u_\tau\}$ is I(0). An example of an I(1) series could be a logarithmic stock price $\{\ln(P_t)\}=\{p_t\}$ where the logarithmic returns $\{\ln(P_t/P_{t-1})\}=\{p_t-p_{t-1}\}$ are approximately I(0). Taking the first difference of an I(1) series allows us to work with the I(0) series, and that is convenient as we have all kinds of useful statistical results derived for stationary series*.

I(2): In case of two unit roots, the first-differenced time series $\{\Delta x_t\}=\{x_t- x_{t-1}\}$ is a cumulative sum of an I(0) time series. In other words, the original series is a "twice-cumulative sum" or "second-order cumulative sum" of an I(0) time series, $x_t=\sum_{\tau=1}^t\left(\sum_{s=1}^\tau v_s\right)$ where $\{v_s\}$ is I(0). Some economists consider the consumer price index to be approximated by I(2) which implies that the price increments are I(1). Indeed, price increments show some persistence, but how good of an approximation I(1) is to them is debatable.

I(3): I am not aware of any I(3) processes in the real world.

If you difference an I($d$) series more than $d$ times, you encounter the problem of overdifferencing. It increases the error variance of the model relative to a model with an appropriate order of differencing (which is $d$) and creates a moving-average component with a negative unit root. That is why differencing should be done with care and not overdone.

*An I(0) time series is quite similar to a stationary one, though there are some subtle differences.