What does the integrated part of the Arima model do?

I understand that the integration is used to 'convert' a non-stationary process into a stationary process using the method of differencing. I'm just unsure exactly what differencing by $$d > 1$$ means.

Say if we have a series $$X_{t}$$ and forecast with $$d>1$$ in the $$\text{ARIMA}(p, d, q)$$ model is this equivalent to forecasting $$Y_{t} = X_{t} - X_{t-1} - \dots - X_{t-d}$$ with $$\text{ARMA}(p, q)$$ or have I misunderstood?

For a fixed $$d$$, you just repeat the process of differentiation $$d$$ times, recursively. For example, if you want to get to $$d = 3$$, start with the first difference:

$$\Delta X_t = X_t - X_{t-1} =: Y_t$$

Then, $$Y_t$$ is just some series, which you can difference just like you did $$X_t$$. That would be the 2nd difference:

$$\Delta^2 X_t = \Delta (\Delta X_t) = \Delta Y_t = Y_t - Y_{t-1} = (X_t - X_{t-1}) - (X_{t-1} - X_{t-2}) = X_t - 2X_{t-1} + X_{t-2}$$

And then once more to get to the third difference:

$$\Delta^3 X_t = \Delta (\Delta^2 X_t) = (X_t - 2X_{t-1} + X_{t-2}) - (X_{t-1} - 2X_{t-2} + X_{t-3}) = X_t - 3X_{t-1} + 3 X_{t-2} - X_{t-3}$$

So, $$\Delta^dX_t$$ is indeed a linear combination of $$X_t, X_{t-1}, ..., X_{t-d}$$, but not the one you suggested.

In an ARIMA(p,d,q) model, $$\Delta^dX_t$$ follows an ARMA(p,q) process, that's correct.